INFO: Started server process [1] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: 172.20.0.1:42836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39214 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38726 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46294 - "GET /node/69bd7f210e93e2314723eb1c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:46302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51220 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36688 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60340 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60514 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49612 - "GET /node/69b35eb5974c91e64560e996%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:49616 - "GET /node/6990ace0c45febfe44521a91%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:33880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50524 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37076 - "GET /node/693b0f09fee8383e92b729e8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40350 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:35:31,935 - lava_callback - INFO - Received callback for node 69cdb9c723eb053795d41f16 2026-04-04 00:35:31,942 - lava_callback - INFO - Starting async job submit for node 69cdb9c723eb053795d41f16 INFO: 172.20.0.1:44488 - "POST /node/69cdb9c723eb053795d41f16 HTTP/1.0" 202 Accepted 2026-04-04 00:35:31,944 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:35:31,969 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9c723eb053795d41f16 HTTP/1.1" 200 None 2026-04-04 00:35:31,970 - lava_callback - DEBUG - Processing node 69cdb9c723eb053795d41f16: baseline-arm-baylibre 2026-04-04 00:35:31,971 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdb9c723eb053795d41f16 2026-04-04 00:35:31,971 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:35:31,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9c723eb053795d41f16 HTTP/1.1" 200 None 2026-04-04 00:35:31,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:35:32,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-04 00:35:32,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:35:32,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdb9c723eb053795d41f16 HTTP/1.1" 200 None 2026-04-04 00:35:32,041 - lava_callback - ERROR - Error processing callback for node 69cdb9c723eb053795d41f16: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44752 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:35:56,801 - lava_callback - INFO - Received callback for node 69d05c7fd2be3e1cf2a5abc5 INFO: 172.20.0.1:44756 - "POST /node/69d05c7fd2be3e1cf2a5abc5 HTTP/1.0" 202 Accepted 2026-04-04 00:35:56,808 - lava_callback - INFO - Starting async job submit for node 69d05c7fd2be3e1cf2a5abc5 2026-04-04 00:35:56,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:35:56,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05c7fd2be3e1cf2a5abc5 HTTP/1.1" 200 None 2026-04-04 00:35:56,846 - lava_callback - DEBUG - Processing node 69d05c7fd2be3e1cf2a5abc5: baseline-arm 2026-04-04 00:35:57,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:35:57,180 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:35:57,182 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d05c7fd2be3e1cf2a5abc5/log.txt.gz 2026-04-04 00:35:57,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:35:57,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:35:57,549 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d05c7fd2be3e1cf2a5abc5/lava-logs.yaml 2026-04-04 00:35:57,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:35:57,689 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:35:57,691 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d05c7fd2be3e1cf2a5abc5/lava_callback.json.gz 2026-04-04 00:35:57,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:35:57,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05c7fd2be3e1cf2a5abc5 HTTP/1.1" 200 None 2026-04-04 00:35:57,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:35:57,744 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cfd2be3e1cf2a5a7f5 HTTP/1.1" 200 None 2026-04-04 00:35:57,746 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:35:57,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05c7fd2be3e1cf2a5abc5 HTTP/1.1" 200 None 2026-04-04 00:35:57,777 - lava_callback - INFO - Completed processing callback for node 69d05c7fd2be3e1cf2a5abc5 2026-04-04 00:36:02,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:36:02,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33436 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:36:11,729 - lava_callback - INFO - Received callback for node 69cdb9ef23eb053795d41f21 INFO: 172.20.0.1:46982 - "POST /node/69cdb9ef23eb053795d41f21 HTTP/1.0" 202 Accepted 2026-04-04 00:36:11,735 - lava_callback - INFO - Starting async job submit for node 69cdb9ef23eb053795d41f21 2026-04-04 00:36:11,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:36:11,752 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9ef23eb053795d41f21 HTTP/1.1" 200 None 2026-04-04 00:36:11,753 - lava_callback - DEBUG - Processing node 69cdb9ef23eb053795d41f21: baseline-arm-baylibre 2026-04-04 00:36:11,753 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdb9ef23eb053795d41f21 2026-04-04 00:36:11,754 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:36:11,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb9ef23eb053795d41f21 HTTP/1.1" 200 None 2026-04-04 00:36:11,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:36:11,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-04 00:36:11,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:36:11,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdb9ef23eb053795d41f21 HTTP/1.1" 200 None 2026-04-04 00:36:11,835 - lava_callback - ERROR - Error processing callback for node 69cdb9ef23eb053795d41f21: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46996 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42324 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:37:51,944 - lava_callback - INFO - Received callback for node 69cdba5223eb053795d41f62 INFO: 172.20.0.1:51472 - "POST /node/69cdba5223eb053795d41f62 HTTP/1.0" 202 Accepted 2026-04-04 00:37:51,952 - lava_callback - INFO - Starting async job submit for node 69cdba5223eb053795d41f62 2026-04-04 00:37:51,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:37:51,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdba5223eb053795d41f62 HTTP/1.1" 200 None 2026-04-04 00:37:51,980 - lava_callback - DEBUG - Processing node 69cdba5223eb053795d41f62: baseline-arm-baylibre 2026-04-04 00:37:51,980 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdba5223eb053795d41f62 2026-04-04 00:37:51,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:37:52,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdba5223eb053795d41f62 HTTP/1.1" 200 None 2026-04-04 00:37:52,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:37:52,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-04 00:37:52,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:37:52,083 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdba5223eb053795d41f62 HTTP/1.1" 200 None 2026-04-04 00:37:52,085 - lava_callback - ERROR - Error processing callback for node 69cdba5223eb053795d41f62: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55686 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53720 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:38:31,824 - lava_callback - INFO - Received callback for node 69cdba7923eb053795d41f83 INFO: 172.20.0.1:48418 - "POST /node/69cdba7923eb053795d41f83 HTTP/1.0" 202 Accepted 2026-04-04 00:38:31,830 - lava_callback - INFO - Starting async job submit for node 69cdba7923eb053795d41f83 2026-04-04 00:38:31,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:38:31,849 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdba7923eb053795d41f83 HTTP/1.1" 200 None 2026-04-04 00:38:31,850 - lava_callback - DEBUG - Processing node 69cdba7923eb053795d41f83: baseline-arm-baylibre 2026-04-04 00:38:31,850 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdba7923eb053795d41f83 2026-04-04 00:38:31,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:38:31,866 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdba7923eb053795d41f83 HTTP/1.1" 200 None 2026-04-04 00:38:31,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:38:31,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-04 00:38:31,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:38:31,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdba7923eb053795d41f83 HTTP/1.1" 200 None 2026-04-04 00:38:31,907 - lava_callback - ERROR - Error processing callback for node 69cdba7923eb053795d41f83: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56178 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47384 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:40:13,659 - lava_callback - INFO - Received callback for node 69cdbadf23eb053795d41f94 INFO: 172.20.0.1:50116 - "POST /node/69cdbadf23eb053795d41f94 HTTP/1.0" 202 Accepted 2026-04-04 00:40:13,665 - lava_callback - INFO - Starting async job submit for node 69cdbadf23eb053795d41f94 2026-04-04 00:40:13,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:40:13,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbadf23eb053795d41f94 HTTP/1.1" 200 None 2026-04-04 00:40:13,680 - lava_callback - DEBUG - Processing node 69cdbadf23eb053795d41f94: baseline-arm-clabbe 2026-04-04 00:40:13,681 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbadf23eb053795d41f94 2026-04-04 00:40:13,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:40:13,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbadf23eb053795d41f94 HTTP/1.1" 200 None 2026-04-04 00:40:13,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:40:13,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-04 00:40:13,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:40:13,732 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbadf23eb053795d41f94 HTTP/1.1" 200 None 2026-04-04 00:40:13,742 - lava_callback - ERROR - Error processing callback for node 69cdbadf23eb053795d41f94: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50120 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56996 - "GET /node/69937a63cebe99dd8e06b9d6%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57010 - "GET /node/694b00fd6c96ed51d3172e99%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59502 - "GET /node/693b0949fee8383e92b71611%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:59510 - "GET /node/6988bc599fc35f969e1ecd93%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:46610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56570 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42170 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:41:52,160 - lava_callback - INFO - Received callback for node 69cdbb4123eb053795d41faf INFO: 172.20.0.1:43920 - "POST /node/69cdbb4123eb053795d41faf HTTP/1.0" 202 Accepted 2026-04-04 00:41:52,169 - lava_callback - INFO - Starting async job submit for node 69cdbb4123eb053795d41faf 2026-04-04 00:41:52,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:41:52,201 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbb4123eb053795d41faf HTTP/1.1" 200 None 2026-04-04 00:41:52,202 - lava_callback - DEBUG - Processing node 69cdbb4123eb053795d41faf: baseline-arm-baylibre 2026-04-04 00:41:52,203 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbb4123eb053795d41faf 2026-04-04 00:41:52,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:41:52,228 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbb4123eb053795d41faf HTTP/1.1" 200 None 2026-04-04 00:41:52,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:41:52,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-04 00:41:52,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:41:52,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbb4123eb053795d41faf HTTP/1.1" 200 None 2026-04-04 00:41:52,302 - lava_callback - ERROR - Error processing callback for node 69cdbb4123eb053795d41faf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 00:41:54,640 - lava_callback - INFO - Received callback for node 69d05e3bd2be3e1cf2a5abf5 INFO: 172.20.0.1:43930 - "POST /node/69d05e3bd2be3e1cf2a5abf5 HTTP/1.0" 202 Accepted 2026-04-04 00:41:54,650 - lava_callback - INFO - Starting async job submit for node 69d05e3bd2be3e1cf2a5abf5 2026-04-04 00:41:54,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:41:54,697 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05e3bd2be3e1cf2a5abf5 HTTP/1.1" 200 None 2026-04-04 00:41:54,698 - lava_callback - DEBUG - Processing node 69d05e3bd2be3e1cf2a5abf5: baseline-x86 2026-04-04 00:41:54,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:41:55,298 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:41:55,301 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3bd2be3e1cf2a5abf5/log.txt.gz 2026-04-04 00:41:55,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:41:55,991 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:41:55,993 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3bd2be3e1cf2a5abf5/lava-logs.yaml 2026-04-04 00:41:56,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:41:56,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:41:56,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3bd2be3e1cf2a5abf5/lava_callback.json.gz 2026-04-04 00:41:56,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:41:56,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05e3bd2be3e1cf2a5abf5 HTTP/1.1" 200 None 2026-04-04 00:41:56,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:41:56,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058e8d2be3e1cf2a5a816 HTTP/1.1" 200 None 2026-04-04 00:41:56,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:41:56,329 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05e3bd2be3e1cf2a5abf5 HTTP/1.1" 200 None 2026-04-04 00:41:56,334 - lava_callback - INFO - Completed processing callback for node 69d05e3bd2be3e1cf2a5abf5 INFO: 172.20.0.1:43944 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:42:02,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:02,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57690 - "GET /node/69862f64177ca0f097bd2413%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57708 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:42:09,877 - lava_callback - INFO - Received callback for node 69d05e39d2be3e1cf2a5abeb INFO: 172.20.0.1:53556 - "POST /node/69d05e39d2be3e1cf2a5abeb HTTP/1.0" 202 Accepted 2026-04-04 00:42:09,887 - lava_callback - INFO - Starting async job submit for node 69d05e39d2be3e1cf2a5abeb 2026-04-04 00:42:09,912 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:09,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05e39d2be3e1cf2a5abeb HTTP/1.1" 200 None 2026-04-04 00:42:09,939 - lava_callback - DEBUG - Processing node 69d05e39d2be3e1cf2a5abeb: baseline-x86 2026-04-04 00:42:10,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:42:10,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:42:10,470 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e39d2be3e1cf2a5abeb/log.txt.gz 2026-04-04 00:42:10,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:42:11,112 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:42:11,114 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e39d2be3e1cf2a5abeb/lava-logs.yaml 2026-04-04 00:42:11,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:42:11,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:42:11,267 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e39d2be3e1cf2a5abeb/lava_callback.json.gz 2026-04-04 00:42:11,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:11,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05e39d2be3e1cf2a5abeb HTTP/1.1" 200 None 2026-04-04 00:42:11,302 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:11,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058e8d2be3e1cf2a5a816 HTTP/1.1" 200 None 2026-04-04 00:42:11,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:11,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05e39d2be3e1cf2a5abeb HTTP/1.1" 200 None 2026-04-04 00:42:11,384 - lava_callback - INFO - Completed processing callback for node 69d05e39d2be3e1cf2a5abeb INFO: 172.20.0.1:53572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59924 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:42:32,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:32,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 00:42:33,720 - lava_callback - INFO - Received callback for node 69cdbb6723eb053795d41fbe INFO: 172.20.0.1:47672 - "POST /node/69cdbb6723eb053795d41fbe HTTP/1.0" 202 Accepted 2026-04-04 00:42:33,728 - lava_callback - INFO - Starting async job submit for node 69cdbb6723eb053795d41fbe 2026-04-04 00:42:33,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:33,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbb6723eb053795d41fbe HTTP/1.1" 200 None 2026-04-04 00:42:33,766 - lava_callback - DEBUG - Processing node 69cdbb6723eb053795d41fbe: baseline-arm-clabbe 2026-04-04 00:42:33,766 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbb6723eb053795d41fbe 2026-04-04 00:42:33,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:33,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbb6723eb053795d41fbe HTTP/1.1" 200 None 2026-04-04 00:42:33,794 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:33,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-04 00:42:33,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:33,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbb6723eb053795d41fbe HTTP/1.1" 200 None 2026-04-04 00:42:33,861 - lava_callback - ERROR - Error processing callback for node 69cdbb6723eb053795d41fbe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38664 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:42:52,641 - lava_callback - INFO - Received callback for node 69d05e3ad2be3e1cf2a5abef INFO: 172.20.0.1:43628 - "POST /node/69d05e3ad2be3e1cf2a5abef HTTP/1.0" 202 Accepted 2026-04-04 00:42:52,650 - lava_callback - INFO - Starting async job submit for node 69d05e3ad2be3e1cf2a5abef 2026-04-04 00:42:52,670 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:52,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05e3ad2be3e1cf2a5abef HTTP/1.1" 200 None 2026-04-04 00:42:52,686 - lava_callback - DEBUG - Processing node 69d05e3ad2be3e1cf2a5abef: baseline-x86 2026-04-04 00:42:53,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:42:53,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:42:53,386 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3ad2be3e1cf2a5abef/log.txt.gz 2026-04-04 00:42:53,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:42:54,196 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:42:54,198 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3ad2be3e1cf2a5abef/lava-logs.yaml 2026-04-04 00:42:54,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:42:54,368 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:42:54,370 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3ad2be3e1cf2a5abef/lava_callback.json.gz 2026-04-04 00:42:54,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:54,448 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05e3ad2be3e1cf2a5abef HTTP/1.1" 200 None 2026-04-04 00:42:54,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:54,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058e8d2be3e1cf2a5a816 HTTP/1.1" 200 None 2026-04-04 00:42:54,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:54,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05e3ad2be3e1cf2a5abef HTTP/1.1" 200 None 2026-04-04 00:42:54,510 - lava_callback - INFO - Completed processing callback for node 69d05e3ad2be3e1cf2a5abef INFO: 172.20.0.1:43640 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:42:57,079 - lava_callback - INFO - Received callback for node 69d05e3cd2be3e1cf2a5abf8 INFO: 172.20.0.1:43652 - "POST /node/69d05e3cd2be3e1cf2a5abf8 HTTP/1.0" 202 Accepted 2026-04-04 00:42:57,086 - lava_callback - INFO - Starting async job submit for node 69d05e3cd2be3e1cf2a5abf8 2026-04-04 00:42:57,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:57,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05e3cd2be3e1cf2a5abf8 HTTP/1.1" 200 None 2026-04-04 00:42:57,120 - lava_callback - DEBUG - Processing node 69d05e3cd2be3e1cf2a5abf8: baseline-x86 2026-04-04 00:42:57,606 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:42:57,834 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:42:57,836 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3cd2be3e1cf2a5abf8/log.txt.gz 2026-04-04 00:42:58,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:42:58,307 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:42:58,310 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3cd2be3e1cf2a5abf8/lava-logs.yaml 2026-04-04 00:42:58,319 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:42:58,667 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:42:58,669 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3cd2be3e1cf2a5abf8/lava_callback.json.gz 2026-04-04 00:42:58,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:58,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05e3cd2be3e1cf2a5abf8 HTTP/1.1" 200 None 2026-04-04 00:42:58,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:58,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058e8d2be3e1cf2a5a816 HTTP/1.1" 200 None 2026-04-04 00:42:58,738 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:42:58,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05e3cd2be3e1cf2a5abf8 HTTP/1.1" 200 None 2026-04-04 00:42:58,771 - lava_callback - INFO - Completed processing callback for node 69d05e3cd2be3e1cf2a5abf8 2026-04-04 00:43:02,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:02,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56138 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:43:12,095 - lava_callback - INFO - Received callback for node 69cdbb8b23eb053795d41fce INFO: 172.20.0.1:39104 - "POST /node/69cdbb8b23eb053795d41fce HTTP/1.0" 202 Accepted 2026-04-04 00:43:12,104 - lava_callback - INFO - Starting async job submit for node 69cdbb8b23eb053795d41fce 2026-04-04 00:43:12,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:12,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbb8b23eb053795d41fce HTTP/1.1" 200 None 2026-04-04 00:43:12,132 - lava_callback - DEBUG - Processing node 69cdbb8b23eb053795d41fce: baseline-arm-baylibre 2026-04-04 00:43:12,133 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbb8b23eb053795d41fce 2026-04-04 00:43:12,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:12,158 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbb8b23eb053795d41fce HTTP/1.1" 200 None 2026-04-04 00:43:12,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:12,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-04 00:43:12,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:12,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbb8b23eb053795d41fce HTTP/1.1" 200 None 2026-04-04 00:43:12,242 - lava_callback - ERROR - Error processing callback for node 69cdbb8b23eb053795d41fce: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39120 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-04 00:43:15,628 - lava_callback - INFO - Received callback for node 69d05e3ad2be3e1cf2a5abf2 INFO: 172.20.0.1:39124 - "POST /node/69d05e3ad2be3e1cf2a5abf2 HTTP/1.0" 202 Accepted 2026-04-04 00:43:15,638 - lava_callback - INFO - Starting async job submit for node 69d05e3ad2be3e1cf2a5abf2 2026-04-04 00:43:15,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:15,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05e3ad2be3e1cf2a5abf2 HTTP/1.1" 200 None 2026-04-04 00:43:15,703 - lava_callback - DEBUG - Processing node 69d05e3ad2be3e1cf2a5abf2: baseline-x86 2026-04-04 00:43:16,013 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:43:16,443 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:43:16,445 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3ad2be3e1cf2a5abf2/log.txt.gz INFO: 172.20.0.1:39138 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:43:16,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:43:17,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:43:17,129 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3ad2be3e1cf2a5abf2/lava-logs.yaml 2026-04-04 00:43:17,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:43:17,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:43:17,385 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-x86-69d05e3ad2be3e1cf2a5abf2/lava_callback.json.gz 2026-04-04 00:43:17,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:17,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05e3ad2be3e1cf2a5abf2 HTTP/1.1" 200 None 2026-04-04 00:43:17,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:17,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058e8d2be3e1cf2a5a816 HTTP/1.1" 200 None 2026-04-04 00:43:17,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:17,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05e3ad2be3e1cf2a5abf2 HTTP/1.1" 200 None 2026-04-04 00:43:17,513 - lava_callback - INFO - Completed processing callback for node 69d05e3ad2be3e1cf2a5abf2 INFO: 172.20.0.1:40746 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:43:32,102 - lava_callback - INFO - Received callback for node 69cdbbac23eb053795d41fda INFO: 172.20.0.1:55688 - "POST /node/69cdbbac23eb053795d41fda HTTP/1.0" 202 Accepted 2026-04-04 00:43:32,109 - lava_callback - INFO - Starting async job submit for node 69cdbbac23eb053795d41fda 2026-04-04 00:43:32,110 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:32,145 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:32,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbbac23eb053795d41fda HTTP/1.1" 200 None 2026-04-04 00:43:32,171 - lava_callback - DEBUG - Processing node 69cdbbac23eb053795d41fda: baseline-arm-baylibre 2026-04-04 00:43:32,171 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbbac23eb053795d41fda 2026-04-04 00:43:32,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:32,186 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 00:43:32,198 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbbac23eb053795d41fda HTTP/1.1" 200 None 2026-04-04 00:43:32,200 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:32,219 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 00:43:32,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:32,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbbac23eb053795d41fda HTTP/1.1" 200 None 2026-04-04 00:43:32,253 - lava_callback - ERROR - Error processing callback for node 69cdbbac23eb053795d41fda: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55706 - "GET /node/693f61ef0d8ce4018da8eb2a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53304 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:43:49,024 - lava_callback - INFO - Received callback for node 69d05eaed2be3e1cf2a5ac41 INFO: 172.20.0.1:53312 - "POST /node/69d05eaed2be3e1cf2a5ac41 HTTP/1.0" 202 Accepted 2026-04-04 00:43:49,034 - lava_callback - INFO - Starting async job submit for node 69d05eaed2be3e1cf2a5ac41 2026-04-04 00:43:49,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:49,061 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eaed2be3e1cf2a5ac41 HTTP/1.1" 200 None 2026-04-04 00:43:49,062 - lava_callback - DEBUG - Processing node 69d05eaed2be3e1cf2a5ac41: baseline-arm 2026-04-04 00:43:49,072 - lava_callback - WARNING - Failed to upload log 2026-04-04 00:43:49,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:43:49,273 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:43:49,275 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eaed2be3e1cf2a5ac41/lava-logs.yaml 2026-04-04 00:43:49,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:43:49,414 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:43:49,417 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eaed2be3e1cf2a5ac41/lava_callback.json.gz 2026-04-04 00:43:49,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:49,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eaed2be3e1cf2a5ac41 HTTP/1.1" 200 None 2026-04-04 00:43:49,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:49,458 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:43:49,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:43:49,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05eaed2be3e1cf2a5ac41 HTTP/1.1" 200 None 2026-04-04 00:43:49,492 - lava_callback - INFO - Completed processing callback for node 69d05eaed2be3e1cf2a5ac41 INFO: 172.20.0.1:45600 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:44:00,299 - lava_callback - INFO - Received callback for node 69d05eb1d2be3e1cf2a5ac52 INFO: 172.20.0.1:37838 - "POST /node/69d05eb1d2be3e1cf2a5ac52 HTTP/1.0" 202 Accepted 2026-04-04 00:44:00,354 - lava_callback - INFO - Starting async job submit for node 69d05eb1d2be3e1cf2a5ac52 2026-04-04 00:44:00,381 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:00,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eb1d2be3e1cf2a5ac52 HTTP/1.1" 200 None 2026-04-04 00:44:00,403 - lava_callback - DEBUG - Processing node 69d05eb1d2be3e1cf2a5ac52: baseline-arm 2026-04-04 00:44:00,651 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:44:01,774 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:01,776 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eb1d2be3e1cf2a5ac52/log.txt.gz 2026-04-04 00:44:01,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:44:02,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:02,130 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eb1d2be3e1cf2a5ac52/lava-logs.yaml 2026-04-04 00:44:02,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:44:02,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:02,239 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 00:44:02,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:02,328 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eb1d2be3e1cf2a5ac52/lava_callback.json.gz 2026-04-04 00:44:02,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:02,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eb1d2be3e1cf2a5ac52 HTTP/1.1" 200 None 2026-04-04 00:44:02,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:02,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:44:02,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:02,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05eb1d2be3e1cf2a5ac52 HTTP/1.1" 200 None 2026-04-04 00:44:02,409 - lava_callback - INFO - Completed processing callback for node 69d05eb1d2be3e1cf2a5ac52 INFO: 172.20.0.1:37842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37856 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:44:12,162 - lava_callback - INFO - Received callback for node 69cdbbd223eb053795d41fe7 INFO: 172.20.0.1:42002 - "POST /node/69cdbbd223eb053795d41fe7 HTTP/1.0" 202 Accepted 2026-04-04 00:44:12,167 - lava_callback - INFO - Starting async job submit for node 69cdbbd223eb053795d41fe7 2026-04-04 00:44:12,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:12,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbbd223eb053795d41fe7 HTTP/1.1" 200 None 2026-04-04 00:44:12,183 - lava_callback - DEBUG - Processing node 69cdbbd223eb053795d41fe7: baseline-arm-baylibre 2026-04-04 00:44:12,183 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbbd223eb053795d41fe7 2026-04-04 00:44:12,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:12,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbbd223eb053795d41fe7 HTTP/1.1" 200 None 2026-04-04 00:44:12,213 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:12,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 00:44:12,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:12,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbbd223eb053795d41fe7 HTTP/1.1" 200 None 2026-04-04 00:44:12,259 - lava_callback - ERROR - Error processing callback for node 69cdbbd223eb053795d41fe7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42014 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:44:18,500 - lava_callback - INFO - Received callback for node 69d05eafd2be3e1cf2a5ac44 INFO: 172.20.0.1:42024 - "POST /node/69d05eafd2be3e1cf2a5ac44 HTTP/1.0" 202 Accepted 2026-04-04 00:44:18,550 - lava_callback - INFO - Starting async job submit for node 69d05eafd2be3e1cf2a5ac44 2026-04-04 00:44:18,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:18,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eafd2be3e1cf2a5ac44 HTTP/1.1" 200 None 2026-04-04 00:44:18,602 - lava_callback - DEBUG - Processing node 69d05eafd2be3e1cf2a5ac44: baseline-arm 2026-04-04 00:44:18,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:44:18,981 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:18,983 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eafd2be3e1cf2a5ac44/log.txt.gz 2026-04-04 00:44:19,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:44:19,263 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:19,265 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eafd2be3e1cf2a5ac44/lava-logs.yaml 2026-04-04 00:44:19,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:44:19,430 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:19,432 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eafd2be3e1cf2a5ac44/lava_callback.json.gz 2026-04-04 00:44:19,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:19,466 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eafd2be3e1cf2a5ac44 HTTP/1.1" 200 None 2026-04-04 00:44:19,468 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:19,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:44:19,484 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:19,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05eafd2be3e1cf2a5ac44 HTTP/1.1" 200 None 2026-04-04 00:44:19,563 - lava_callback - INFO - Completed processing callback for node 69d05eafd2be3e1cf2a5ac44 INFO: 172.20.0.1:49620 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:44:32,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:32,262 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 00:44:36,133 - lava_callback - INFO - Received callback for node 69d05eb0d2be3e1cf2a5ac4f INFO: 172.20.0.1:46356 - "POST /node/69d05eb0d2be3e1cf2a5ac4f HTTP/1.0" 202 Accepted 2026-04-04 00:44:36,143 - lava_callback - INFO - Starting async job submit for node 69d05eb0d2be3e1cf2a5ac4f 2026-04-04 00:44:36,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:36,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eb0d2be3e1cf2a5ac4f HTTP/1.1" 200 None 2026-04-04 00:44:36,192 - lava_callback - DEBUG - Processing node 69d05eb0d2be3e1cf2a5ac4f: baseline-arm 2026-04-04 00:44:36,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:46372 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:44:36,531 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:36,533 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eb0d2be3e1cf2a5ac4f/log.txt.gz 2026-04-04 00:44:36,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:44:36,857 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:36,859 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eb0d2be3e1cf2a5ac4f/lava-logs.yaml 2026-04-04 00:44:36,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:44:37,036 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:37,037 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eb0d2be3e1cf2a5ac4f/lava_callback.json.gz 2026-04-04 00:44:37,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:37,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eb0d2be3e1cf2a5ac4f HTTP/1.1" 200 None 2026-04-04 00:44:37,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:37,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:44:37,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:37,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05eb0d2be3e1cf2a5ac4f HTTP/1.1" 200 None 2026-04-04 00:44:37,135 - lava_callback - INFO - Completed processing callback for node 69d05eb0d2be3e1cf2a5ac4f INFO: 172.20.0.1:55316 - "GET /node/69a15fc2a41e9451dcf411e3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55332 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:44:52,221 - lava_callback - INFO - Received callback for node 69cdbbfa23eb053795d41ff3 INFO: 172.20.0.1:44656 - "POST /node/69cdbbfa23eb053795d41ff3 HTTP/1.0" 202 Accepted 2026-04-04 00:44:52,227 - lava_callback - INFO - Starting async job submit for node 69cdbbfa23eb053795d41ff3 2026-04-04 00:44:52,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:52,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbbfa23eb053795d41ff3 HTTP/1.1" 200 None 2026-04-04 00:44:52,243 - lava_callback - DEBUG - Processing node 69cdbbfa23eb053795d41ff3: baseline-arm-baylibre 2026-04-04 00:44:52,243 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbbfa23eb053795d41ff3 2026-04-04 00:44:52,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:52,257 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbbfa23eb053795d41ff3 HTTP/1.1" 200 None 2026-04-04 00:44:52,259 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:52,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 00:44:52,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:52,294 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbbfa23eb053795d41ff3 HTTP/1.1" 200 None 2026-04-04 00:44:52,303 - lava_callback - ERROR - Error processing callback for node 69cdbbfa23eb053795d41ff3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44658 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:44:57,331 - lava_callback - INFO - Received callback for node 69d05ec1d2be3e1cf2a5ac7e INFO: 172.20.0.1:44670 - "POST /node/69d05ec1d2be3e1cf2a5ac7e HTTP/1.0" 202 Accepted 2026-04-04 00:44:57,337 - lava_callback - INFO - Starting async job submit for node 69d05ec1d2be3e1cf2a5ac7e 2026-04-04 00:44:57,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:57,398 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ec1d2be3e1cf2a5ac7e HTTP/1.1" 200 None 2026-04-04 00:44:57,399 - lava_callback - DEBUG - Processing node 69d05ec1d2be3e1cf2a5ac7e: baseline-arm-broonie 2026-04-04 00:44:57,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:44:57,814 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:57,816 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ec1d2be3e1cf2a5ac7e/log.txt.gz 2026-04-04 00:44:57,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:44:58,167 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:58,169 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ec1d2be3e1cf2a5ac7e/lava-logs.yaml 2026-04-04 00:44:58,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:44:58,348 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:44:58,351 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ec1d2be3e1cf2a5ac7e/lava_callback.json.gz 2026-04-04 00:44:58,387 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:58,401 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ec1d2be3e1cf2a5ac7e HTTP/1.1" 200 None 2026-04-04 00:44:58,402 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:58,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:44:58,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:44:58,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ec1d2be3e1cf2a5ac7e HTTP/1.1" 200 None 2026-04-04 00:44:58,472 - lava_callback - INFO - Completed processing callback for node 69d05ec1d2be3e1cf2a5ac7e 2026-04-04 00:45:02,264 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:02,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47888 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:45:08,218 - lava_callback - INFO - Received callback for node 69d05ed6d2be3e1cf2a5acf7 INFO: 172.20.0.1:47890 - "POST /node/69d05ed6d2be3e1cf2a5acf7 HTTP/1.0" 202 Accepted 2026-04-04 00:45:08,265 - lava_callback - INFO - Starting async job submit for node 69d05ed6d2be3e1cf2a5acf7 2026-04-04 00:45:08,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:08,300 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ed6d2be3e1cf2a5acf7 HTTP/1.1" 200 None 2026-04-04 00:45:08,300 - lava_callback - DEBUG - Processing node 69d05ed6d2be3e1cf2a5acf7: baseline-arm-pengutronix 2026-04-04 00:45:08,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:08,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:08,527 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed6d2be3e1cf2a5acf7/log.txt.gz 2026-04-04 00:45:08,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:08,759 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:08,761 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed6d2be3e1cf2a5acf7/lava-logs.yaml 2026-04-04 00:45:08,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:08,881 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:08,883 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed6d2be3e1cf2a5acf7/lava_callback.json.gz 2026-04-04 00:45:08,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:08,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ed6d2be3e1cf2a5acf7 HTTP/1.1" 200 None 2026-04-04 00:45:08,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:08,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:45:08,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:08,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ed6d2be3e1cf2a5acf7 HTTP/1.1" 200 None 2026-04-04 00:45:08,984 - lava_callback - INFO - Completed processing callback for node 69d05ed6d2be3e1cf2a5acf7 2026-04-04 00:45:11,364 - lava_callback - INFO - Received callback for node 69d05ed3d2be3e1cf2a5acf1 INFO: 172.20.0.1:42974 - "POST /node/69d05ed3d2be3e1cf2a5acf1 HTTP/1.0" 202 Accepted 2026-04-04 00:45:11,373 - lava_callback - INFO - Starting async job submit for node 69d05ed3d2be3e1cf2a5acf1 2026-04-04 00:45:11,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:11,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ed3d2be3e1cf2a5acf1 HTTP/1.1" 200 None 2026-04-04 00:45:11,419 - lava_callback - DEBUG - Processing node 69d05ed3d2be3e1cf2a5acf1: baseline-arm-pengutronix 2026-04-04 00:45:11,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:11,677 - lava_callback - INFO - Received callback for node 69d05eadd2be3e1cf2a5ac3e 2026-04-04 00:45:11,684 - lava_callback - INFO - Starting async job submit for node 69d05eadd2be3e1cf2a5ac3e INFO: 172.20.0.1:42988 - "POST /node/69d05eadd2be3e1cf2a5ac3e HTTP/1.0" 202 Accepted 2026-04-04 00:45:11,704 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:11,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:11,709 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed3d2be3e1cf2a5acf1/log.txt.gz 2026-04-04 00:45:11,773 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eadd2be3e1cf2a5ac3e HTTP/1.1" 200 None 2026-04-04 00:45:11,784 - lava_callback - DEBUG - Processing node 69d05eadd2be3e1cf2a5ac3e: baseline-arm 2026-04-04 00:45:11,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:12,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:12,174 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:12,176 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eadd2be3e1cf2a5ac3e/log.txt.gz 2026-04-04 00:45:12,188 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:12,222 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed3d2be3e1cf2a5acf1/lava-logs.yaml 2026-04-04 00:45:12,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:12,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:12,486 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:12,489 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed3d2be3e1cf2a5acf1/lava_callback.json.gz 2026-04-04 00:45:12,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:12,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ed3d2be3e1cf2a5acf1 HTTP/1.1" 200 None 2026-04-04 00:45:12,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:12,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:45:12,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:12,558 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:12,561 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eadd2be3e1cf2a5ac3e/lava-logs.yaml 2026-04-04 00:45:12,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:12,573 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ed3d2be3e1cf2a5acf1 HTTP/1.1" 200 None 2026-04-04 00:45:12,575 - lava_callback - INFO - Completed processing callback for node 69d05ed3d2be3e1cf2a5acf1 2026-04-04 00:45:12,789 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:12,790 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eadd2be3e1cf2a5ac3e/lava_callback.json.gz 2026-04-04 00:45:12,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:12,831 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eadd2be3e1cf2a5ac3e HTTP/1.1" 200 None 2026-04-04 00:45:12,832 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:12,846 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:45:12,847 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:12,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05eadd2be3e1cf2a5ac3e HTTP/1.1" 200 None 2026-04-04 00:45:12,872 - lava_callback - INFO - Completed processing callback for node 69d05eadd2be3e1cf2a5ac3e 2026-04-04 00:45:13,478 - lava_callback - INFO - Received callback for node 69d05ed2d2be3e1cf2a5acee INFO: 172.20.0.1:42992 - "POST /node/69d05ed2d2be3e1cf2a5acee HTTP/1.0" 202 Accepted 2026-04-04 00:45:13,484 - lava_callback - INFO - Starting async job submit for node 69d05ed2d2be3e1cf2a5acee 2026-04-04 00:45:13,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:13,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ed2d2be3e1cf2a5acee HTTP/1.1" 200 None 2026-04-04 00:45:13,536 - lava_callback - DEBUG - Processing node 69d05ed2d2be3e1cf2a5acee: baseline-arm-pengutronix 2026-04-04 00:45:13,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:13,863 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:13,865 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed2d2be3e1cf2a5acee/log.txt.gz 2026-04-04 00:45:13,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:14,173 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:14,175 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed2d2be3e1cf2a5acee/lava-logs.yaml 2026-04-04 00:45:14,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:14,367 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:14,369 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed2d2be3e1cf2a5acee/lava_callback.json.gz 2026-04-04 00:45:14,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:14,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ed2d2be3e1cf2a5acee HTTP/1.1" 200 None 2026-04-04 00:45:14,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:14,441 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:45:14,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:14,470 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ed2d2be3e1cf2a5acee HTTP/1.1" 200 None 2026-04-04 00:45:14,471 - lava_callback - INFO - Completed processing callback for node 69d05ed2d2be3e1cf2a5acee INFO: 172.20.0.1:42994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42996 - "GET /node/6992d95631166481c8dd3fc8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:43004 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:45:17,795 - lava_callback - INFO - Received callback for node 69d05ebfd2be3e1cf2a5ac7b INFO: 172.20.0.1:43020 - "POST /node/69d05ebfd2be3e1cf2a5ac7b HTTP/1.0" 202 Accepted 2026-04-04 00:45:17,802 - lava_callback - INFO - Starting async job submit for node 69d05ebfd2be3e1cf2a5ac7b 2026-04-04 00:45:17,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:17,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ebfd2be3e1cf2a5ac7b HTTP/1.1" 200 None 2026-04-04 00:45:17,843 - lava_callback - DEBUG - Processing node 69d05ebfd2be3e1cf2a5ac7b: baseline-arm-broonie 2026-04-04 00:45:18,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:18,229 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:18,231 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ebfd2be3e1cf2a5ac7b/log.txt.gz 2026-04-04 00:45:18,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:18,452 - lava_callback - INFO - Received callback for node 69d05ec2d2be3e1cf2a5ac81 INFO: 172.20.0.1:43024 - "POST /node/69d05ec2d2be3e1cf2a5ac81 HTTP/1.0" 202 Accepted 2026-04-04 00:45:18,458 - lava_callback - INFO - Starting async job submit for node 69d05ec2d2be3e1cf2a5ac81 2026-04-04 00:45:18,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:18,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ec2d2be3e1cf2a5ac81 HTTP/1.1" 200 None 2026-04-04 00:45:18,527 - lava_callback - DEBUG - Processing node 69d05ec2d2be3e1cf2a5ac81: baseline-arm-broonie 2026-04-04 00:45:18,612 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:18,640 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ebfd2be3e1cf2a5ac7b/lava-logs.yaml 2026-04-04 00:45:18,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:18,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:18,997 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:18,999 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ec2d2be3e1cf2a5ac81/log.txt.gz 2026-04-04 00:45:19,012 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:19,052 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ebfd2be3e1cf2a5ac7b/lava_callback.json.gz 2026-04-04 00:45:19,123 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:19,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ebfd2be3e1cf2a5ac7b HTTP/1.1" 200 None 2026-04-04 00:45:19,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:19,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:19,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:45:19,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:19,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ebfd2be3e1cf2a5ac7b HTTP/1.1" 200 None 2026-04-04 00:45:19,265 - lava_callback - INFO - Completed processing callback for node 69d05ebfd2be3e1cf2a5ac7b 2026-04-04 00:45:19,485 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:19,488 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ec2d2be3e1cf2a5ac81/lava-logs.yaml 2026-04-04 00:45:19,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:19,647 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:19,649 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ec2d2be3e1cf2a5ac81/lava_callback.json.gz 2026-04-04 00:45:19,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:19,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ec2d2be3e1cf2a5ac81 HTTP/1.1" 200 None 2026-04-04 00:45:19,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:19,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:45:19,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:19,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ec2d2be3e1cf2a5ac81 HTTP/1.1" 200 None 2026-04-04 00:45:19,757 - lava_callback - INFO - Completed processing callback for node 69d05ec2d2be3e1cf2a5ac81 INFO: 172.20.0.1:49400 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:45:28,147 - lava_callback - INFO - Received callback for node 69d05ec7d2be3e1cf2a5ac99 INFO: 172.20.0.1:49416 - "POST /node/69d05ec7d2be3e1cf2a5ac99 HTTP/1.0" 202 Accepted 2026-04-04 00:45:28,153 - lava_callback - INFO - Starting async job submit for node 69d05ec7d2be3e1cf2a5ac99 2026-04-04 00:45:28,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:28,195 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ec7d2be3e1cf2a5ac99 HTTP/1.1" 200 None 2026-04-04 00:45:28,196 - lava_callback - DEBUG - Processing node 69d05ec7d2be3e1cf2a5ac99: baseline-arm-clabbe 2026-04-04 00:45:28,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:28,520 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:28,522 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69d05ec7d2be3e1cf2a5ac99/log.txt.gz 2026-04-04 00:45:28,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:28,833 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:28,835 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69d05ec7d2be3e1cf2a5ac99/lava-logs.yaml 2026-04-04 00:45:28,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:28,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:28,972 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69d05ec7d2be3e1cf2a5ac99/lava_callback.json.gz 2026-04-04 00:45:28,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:29,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ec7d2be3e1cf2a5ac99 HTTP/1.1" 200 None 2026-04-04 00:45:29,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:29,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:45:29,028 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:29,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ec7d2be3e1cf2a5ac99 HTTP/1.1" 200 None 2026-04-04 00:45:29,061 - lava_callback - INFO - Completed processing callback for node 69d05ec7d2be3e1cf2a5ac99 2026-04-04 00:45:32,182 - lava_callback - INFO - Received callback for node 69cdbc1c23eb053795d42021 INFO: 172.20.0.1:58392 - "POST /node/69cdbc1c23eb053795d42021 HTTP/1.0" 202 Accepted 2026-04-04 00:45:32,188 - lava_callback - INFO - Starting async job submit for node 69cdbc1c23eb053795d42021 2026-04-04 00:45:32,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:32,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1c23eb053795d42021 HTTP/1.1" 200 None 2026-04-04 00:45:32,224 - lava_callback - DEBUG - Processing node 69cdbc1c23eb053795d42021: baseline-arm-baylibre 2026-04-04 00:45:32,224 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc1c23eb053795d42021 2026-04-04 00:45:32,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:32,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1c23eb053795d42021 HTTP/1.1" 200 None 2026-04-04 00:45:32,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:32,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:32,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:32,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc1c23eb053795d42021 HTTP/1.1" 200 None 2026-04-04 00:45:32,279 - lava_callback - ERROR - Error processing callback for node 69cdbc1c23eb053795d42021: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 00:45:32,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:32,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-04 00:45:32,849 - lava_callback - INFO - Received callback for node 69cdbc1e23eb053795d42024 INFO: 172.20.0.1:58402 - "POST /node/69cdbc1e23eb053795d42024 HTTP/1.0" 202 Accepted 2026-04-04 00:45:32,854 - lava_callback - INFO - Starting async job submit for node 69cdbc1e23eb053795d42024 2026-04-04 00:45:32,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:32,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1e23eb053795d42024 HTTP/1.1" 200 None 2026-04-04 00:45:32,870 - lava_callback - DEBUG - Processing node 69cdbc1e23eb053795d42024: baseline-arm-baylibre 2026-04-04 00:45:32,870 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc1e23eb053795d42024 2026-04-04 00:45:32,871 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:32,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1e23eb053795d42024 HTTP/1.1" 200 None 2026-04-04 00:45:32,892 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:32,906 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:32,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:32,928 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc1e23eb053795d42024 HTTP/1.1" 200 None 2026-04-04 00:45:32,930 - lava_callback - ERROR - Error processing callback for node 69cdbc1e23eb053795d42024: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 00:45:33,513 - lava_callback - INFO - Received callback for node 69cdbc1f23eb053795d42027 INFO: 172.20.0.1:58410 - "POST /node/69cdbc1f23eb053795d42027 HTTP/1.0" 202 Accepted 2026-04-04 00:45:33,520 - lava_callback - INFO - Starting async job submit for node 69cdbc1f23eb053795d42027 2026-04-04 00:45:33,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:33,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1f23eb053795d42027 HTTP/1.1" 200 None 2026-04-04 00:45:33,538 - lava_callback - DEBUG - Processing node 69cdbc1f23eb053795d42027: baseline-arm-baylibre 2026-04-04 00:45:33,538 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc1f23eb053795d42027 2026-04-04 00:45:33,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:33,560 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc1f23eb053795d42027 HTTP/1.1" 200 None 2026-04-04 00:45:33,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:33,575 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:33,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:33,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc1f23eb053795d42027 HTTP/1.1" 200 None 2026-04-04 00:45:33,614 - lava_callback - ERROR - Error processing callback for node 69cdbc1f23eb053795d42027: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 00:45:34,172 - lava_callback - INFO - Received callback for node 69cdbc2023eb053795d4202b INFO: 172.20.0.1:58412 - "POST /node/69cdbc2023eb053795d4202b HTTP/1.0" 202 Accepted 2026-04-04 00:45:34,178 - lava_callback - INFO - Starting async job submit for node 69cdbc2023eb053795d4202b 2026-04-04 00:45:34,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:34,194 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2023eb053795d4202b HTTP/1.1" 200 None 2026-04-04 00:45:34,195 - lava_callback - DEBUG - Processing node 69cdbc2023eb053795d4202b: baseline-arm-baylibre 2026-04-04 00:45:34,196 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc2023eb053795d4202b 2026-04-04 00:45:34,196 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:34,215 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2023eb053795d4202b HTTP/1.1" 200 None 2026-04-04 00:45:34,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:34,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:34,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:34,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc2023eb053795d4202b HTTP/1.1" 200 None 2026-04-04 00:45:34,270 - lava_callback - ERROR - Error processing callback for node 69cdbc2023eb053795d4202b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 00:45:34,842 - lava_callback - INFO - Received callback for node 69cdbc2323eb053795d42038 INFO: 172.20.0.1:58416 - "POST /node/69cdbc2323eb053795d42038 HTTP/1.0" 202 Accepted 2026-04-04 00:45:34,851 - lava_callback - INFO - Starting async job submit for node 69cdbc2323eb053795d42038 2026-04-04 00:45:34,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:34,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2323eb053795d42038 HTTP/1.1" 200 None 2026-04-04 00:45:34,874 - lava_callback - DEBUG - Processing node 69cdbc2323eb053795d42038: baseline-arm-baylibre 2026-04-04 00:45:34,874 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc2323eb053795d42038 2026-04-04 00:45:34,875 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:34,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2323eb053795d42038 HTTP/1.1" 200 None 2026-04-04 00:45:34,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:34,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:34,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:34,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc2323eb053795d42038 HTTP/1.1" 200 None 2026-04-04 00:45:34,950 - lava_callback - ERROR - Error processing callback for node 69cdbc2323eb053795d42038: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 00:45:35,528 - lava_callback - INFO - Received callback for node 69cdbc2523eb053795d4203b INFO: 172.20.0.1:58418 - "POST /node/69cdbc2523eb053795d4203b HTTP/1.0" 202 Accepted 2026-04-04 00:45:35,534 - lava_callback - INFO - Starting async job submit for node 69cdbc2523eb053795d4203b 2026-04-04 00:45:35,535 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:35,549 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2523eb053795d4203b HTTP/1.1" 200 None 2026-04-04 00:45:35,550 - lava_callback - DEBUG - Processing node 69cdbc2523eb053795d4203b: baseline-arm-baylibre 2026-04-04 00:45:35,550 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc2523eb053795d4203b 2026-04-04 00:45:35,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:35,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2523eb053795d4203b HTTP/1.1" 200 None 2026-04-04 00:45:35,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:35,579 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:35,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:35,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc2523eb053795d4203b HTTP/1.1" 200 None 2026-04-04 00:45:35,633 - lava_callback - ERROR - Error processing callback for node 69cdbc2523eb053795d4203b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58420 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:45:38,329 - lava_callback - INFO - Received callback for node 69d05eced2be3e1cf2a5acdb INFO: 172.20.0.1:58434 - "POST /node/69d05eced2be3e1cf2a5acdb HTTP/1.0" 202 Accepted 2026-04-04 00:45:38,336 - lava_callback - INFO - Starting async job submit for node 69d05eced2be3e1cf2a5acdb 2026-04-04 00:45:38,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:38,386 - lava_callback - INFO - Received callback for node 69d05ebed2be3e1cf2a5ac76 INFO: 172.20.0.1:58446 - "POST /node/69d05ebed2be3e1cf2a5ac76 HTTP/1.0" 202 Accepted 2026-04-04 00:45:38,392 - lava_callback - INFO - Starting async job submit for node 69d05ebed2be3e1cf2a5ac76 2026-04-04 00:45:38,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eced2be3e1cf2a5acdb HTTP/1.1" 200 None 2026-04-04 00:45:38,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:38,427 - lava_callback - DEBUG - Processing node 69d05eced2be3e1cf2a5acdb: baseline-arm-kontron 2026-04-04 00:45:38,493 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ebed2be3e1cf2a5ac76 HTTP/1.1" 200 None 2026-04-04 00:45:38,504 - lava_callback - DEBUG - Processing node 69d05ebed2be3e1cf2a5ac76: baseline-arm-broonie 2026-04-04 00:45:38,928 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:38,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:39,053 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:39,055 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d05eced2be3e1cf2a5acdb/log.txt.gz 2026-04-04 00:45:39,133 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:39,146 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ebed2be3e1cf2a5ac76/log.txt.gz 2026-04-04 00:45:39,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:39,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:39,556 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:39,558 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d05eced2be3e1cf2a5acdb/lava-logs.yaml 2026-04-04 00:45:39,560 - lava_callback - INFO - Received callback for node 69d05ecfd2be3e1cf2a5acde 2026-04-04 00:45:39,569 - lava_callback - INFO - Starting async job submit for node 69d05ecfd2be3e1cf2a5acde INFO: 172.20.0.1:58450 - "POST /node/69d05ecfd2be3e1cf2a5acde HTTP/1.0" 202 Accepted 2026-04-04 00:45:39,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:39,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:39,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:39,598 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ebed2be3e1cf2a5ac76/lava-logs.yaml 2026-04-04 00:45:39,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:39,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ecfd2be3e1cf2a5acde HTTP/1.1" 200 None 2026-04-04 00:45:39,622 - lava_callback - DEBUG - Processing node 69d05ecfd2be3e1cf2a5acde: baseline-arm-kontron 2026-04-04 00:45:39,801 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:39,803 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d05eced2be3e1cf2a5acdb/lava_callback.json.gz 2026-04-04 00:45:39,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:39,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:39,923 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-broonie-69d05ebed2be3e1cf2a5ac76/lava_callback.json.gz 2026-04-04 00:45:39,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:39,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:39,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eced2be3e1cf2a5acdb HTTP/1.1" 200 None 2026-04-04 00:45:39,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:39,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ebed2be3e1cf2a5ac76 HTTP/1.1" 200 None 2026-04-04 00:45:39,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:39,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:45:39,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:40,018 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:45:40,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:40,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05eced2be3e1cf2a5acdb HTTP/1.1" 200 None 2026-04-04 00:45:40,037 - lava_callback - INFO - Completed processing callback for node 69d05eced2be3e1cf2a5acdb 2026-04-04 00:45:40,108 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ebed2be3e1cf2a5ac76 HTTP/1.1" 200 None 2026-04-04 00:45:40,110 - lava_callback - INFO - Completed processing callback for node 69d05ebed2be3e1cf2a5ac76 2026-04-04 00:45:40,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:40,239 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d05ecfd2be3e1cf2a5acde/log.txt.gz 2026-04-04 00:45:40,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:40,594 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:40,596 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d05ecfd2be3e1cf2a5acde/lava-logs.yaml 2026-04-04 00:45:40,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:45:40,728 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:45:40,730 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-kontron-69d05ecfd2be3e1cf2a5acde/lava_callback.json.gz 2026-04-04 00:45:40,753 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:40,769 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ecfd2be3e1cf2a5acde HTTP/1.1" 200 None 2026-04-04 00:45:40,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:40,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:45:40,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:40,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ecfd2be3e1cf2a5acde HTTP/1.1" 200 None 2026-04-04 00:45:40,812 - lava_callback - INFO - Completed processing callback for node 69d05ecfd2be3e1cf2a5acde INFO: 172.20.0.1:37378 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:45:53,804 - lava_callback - INFO - Received callback for node 69cdbc2c23eb053795d42058 INFO: 172.20.0.1:49624 - "POST /node/69cdbc2c23eb053795d42058 HTTP/1.0" 202 Accepted 2026-04-04 00:45:53,812 - lava_callback - INFO - Starting async job submit for node 69cdbc2c23eb053795d42058 2026-04-04 00:45:53,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:53,888 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2c23eb053795d42058 HTTP/1.1" 200 None 2026-04-04 00:45:53,889 - lava_callback - DEBUG - Processing node 69cdbc2c23eb053795d42058: baseline-arm-clabbe 2026-04-04 00:45:53,889 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc2c23eb053795d42058 2026-04-04 00:45:53,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:53,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2c23eb053795d42058 HTTP/1.1" 200 None 2026-04-04 00:45:53,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:53,925 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:53,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:53,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc2c23eb053795d42058 HTTP/1.1" 200 None 2026-04-04 00:45:53,960 - lava_callback - ERROR - Error processing callback for node 69cdbc2c23eb053795d42058: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 00:45:54,892 - lava_callback - INFO - Received callback for node 69cdbc2e23eb053795d4205e INFO: 172.20.0.1:49632 - "POST /node/69cdbc2e23eb053795d4205e HTTP/1.0" 202 Accepted 2026-04-04 00:45:54,898 - lava_callback - INFO - Starting async job submit for node 69cdbc2e23eb053795d4205e 2026-04-04 00:45:54,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:54,914 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2e23eb053795d4205e HTTP/1.1" 200 None 2026-04-04 00:45:54,915 - lava_callback - DEBUG - Processing node 69cdbc2e23eb053795d4205e: baseline-arm-clabbe 2026-04-04 00:45:54,915 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc2e23eb053795d4205e 2026-04-04 00:45:54,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:54,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc2e23eb053795d4205e HTTP/1.1" 200 None 2026-04-04 00:45:54,932 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:54,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:54,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:54,980 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc2e23eb053795d4205e HTTP/1.1" 200 None 2026-04-04 00:45:54,982 - lava_callback - ERROR - Error processing callback for node 69cdbc2e23eb053795d4205e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 00:45:55,550 - lava_callback - INFO - Received callback for node 69cdbc3023eb053795d42065 INFO: 172.20.0.1:49646 - "POST /node/69cdbc3023eb053795d42065 HTTP/1.0" 202 Accepted 2026-04-04 00:45:55,556 - lava_callback - INFO - Starting async job submit for node 69cdbc3023eb053795d42065 2026-04-04 00:45:55,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:55,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3023eb053795d42065 HTTP/1.1" 200 None 2026-04-04 00:45:55,572 - lava_callback - DEBUG - Processing node 69cdbc3023eb053795d42065: baseline-arm-clabbe 2026-04-04 00:45:55,572 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc3023eb053795d42065 2026-04-04 00:45:55,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:55,591 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3023eb053795d42065 HTTP/1.1" 200 None 2026-04-04 00:45:55,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:55,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:55,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:55,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc3023eb053795d42065 HTTP/1.1" 200 None 2026-04-04 00:45:55,643 - lava_callback - ERROR - Error processing callback for node 69cdbc3023eb053795d42065: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 00:45:56,262 - lava_callback - INFO - Received callback for node 69cdbc3123eb053795d42068 INFO: 172.20.0.1:49652 - "POST /node/69cdbc3123eb053795d42068 HTTP/1.0" 202 Accepted 2026-04-04 00:45:56,268 - lava_callback - INFO - Starting async job submit for node 69cdbc3123eb053795d42068 2026-04-04 00:45:56,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:56,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3123eb053795d42068 HTTP/1.1" 200 None 2026-04-04 00:45:56,288 - lava_callback - DEBUG - Processing node 69cdbc3123eb053795d42068: baseline-arm-clabbe 2026-04-04 00:45:56,288 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc3123eb053795d42068 2026-04-04 00:45:56,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:56,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3123eb053795d42068 HTTP/1.1" 200 None 2026-04-04 00:45:56,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:56,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:56,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:56,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc3123eb053795d42068 HTTP/1.1" 200 None 2026-04-04 00:45:56,345 - lava_callback - ERROR - Error processing callback for node 69cdbc3123eb053795d42068: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49664 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:45:56,950 - lava_callback - INFO - Received callback for node 69cdbc3223eb053795d4206c INFO: 172.20.0.1:49680 - "POST /node/69cdbc3223eb053795d4206c HTTP/1.0" 202 Accepted 2026-04-04 00:45:56,955 - lava_callback - INFO - Starting async job submit for node 69cdbc3223eb053795d4206c 2026-04-04 00:45:56,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:56,974 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3223eb053795d4206c HTTP/1.1" 200 None 2026-04-04 00:45:56,975 - lava_callback - DEBUG - Processing node 69cdbc3223eb053795d4206c: baseline-arm-clabbe 2026-04-04 00:45:56,975 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc3223eb053795d4206c 2026-04-04 00:45:56,976 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:57,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3223eb053795d4206c HTTP/1.1" 200 None 2026-04-04 00:45:57,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:57,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:57,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:57,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc3223eb053795d4206c HTTP/1.1" 200 None 2026-04-04 00:45:57,049 - lava_callback - ERROR - Error processing callback for node 69cdbc3223eb053795d4206c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 00:45:57,656 - lava_callback - INFO - Received callback for node 69cdbc3423eb053795d42075 INFO: 172.20.0.1:49682 - "POST /node/69cdbc3423eb053795d42075 HTTP/1.0" 202 Accepted 2026-04-04 00:45:57,662 - lava_callback - INFO - Starting async job submit for node 69cdbc3423eb053795d42075 2026-04-04 00:45:57,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:57,683 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3423eb053795d42075 HTTP/1.1" 200 None 2026-04-04 00:45:57,684 - lava_callback - DEBUG - Processing node 69cdbc3423eb053795d42075: baseline-arm-clabbe 2026-04-04 00:45:57,685 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc3423eb053795d42075 2026-04-04 00:45:57,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:57,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc3423eb053795d42075 HTTP/1.1" 200 None 2026-04-04 00:45:57,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:57,719 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 00:45:57,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:45:57,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc3423eb053795d42075 HTTP/1.1" 200 None 2026-04-04 00:45:57,757 - lava_callback - ERROR - Error processing callback for node 69cdbc3423eb053795d42075: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 00:46:02,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:46:02,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34170 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:46:11,682 - lava_callback - INFO - Received callback for node 69d05ee9d2be3e1cf2a5ad46 INFO: 172.20.0.1:49624 - "POST /node/69d05ee9d2be3e1cf2a5ad46 HTTP/1.0" 202 Accepted 2026-04-04 00:46:11,693 - lava_callback - INFO - Starting async job submit for node 69d05ee9d2be3e1cf2a5ad46 2026-04-04 00:46:11,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:46:11,731 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ee9d2be3e1cf2a5ad46 HTTP/1.1" 200 None 2026-04-04 00:46:11,732 - lava_callback - DEBUG - Processing node 69d05ee9d2be3e1cf2a5ad46: ltp-dio 2026-04-04 00:46:12,126 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:46:12,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:46:12,314 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-dio-69d05ee9d2be3e1cf2a5ad46/log.txt.gz 2026-04-04 00:46:12,536 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:46:12,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:46:12,769 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-dio-69d05ee9d2be3e1cf2a5ad46/lava-logs.yaml 2026-04-04 00:46:12,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:46:12,950 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:46:12,952 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-dio-69d05ee9d2be3e1cf2a5ad46/lava_callback.json.gz 2026-04-04 00:46:12,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:46:13,013 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ee9d2be3e1cf2a5ad46 HTTP/1.1" 200 None 2026-04-04 00:46:13,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:46:13,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:46:13,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:46:13,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ee9d2be3e1cf2a5ad46 HTTP/1.1" 200 None 2026-04-04 00:46:13,064 - lava_callback - INFO - Completed processing callback for node 69d05ee9d2be3e1cf2a5ad46 INFO: 172.20.0.1:49636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37868 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:46:32,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:46:32,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51274 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:46:54,981 - lava_callback - INFO - Received callback for node 69d05f66d2be3e1cf2a5ae3d INFO: 172.20.0.1:55974 - "POST /node/69d05f66d2be3e1cf2a5ae3d HTTP/1.0" 202 Accepted 2026-04-04 00:46:54,990 - lava_callback - INFO - Starting async job submit for node 69d05f66d2be3e1cf2a5ae3d 2026-04-04 00:46:54,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:46:55,026 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05f66d2be3e1cf2a5ae3d HTTP/1.1" 200 None 2026-04-04 00:46:55,027 - lava_callback - DEBUG - Processing node 69d05f66d2be3e1cf2a5ae3d: baseline-arm 2026-04-04 00:46:55,036 - lava_callback - WARNING - Failed to upload log 2026-04-04 00:46:55,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:46:55,214 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:46:55,216 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d05f66d2be3e1cf2a5ae3d/lava-logs.yaml 2026-04-04 00:46:55,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:46:55,351 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:46:55,354 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d05f66d2be3e1cf2a5ae3d/lava_callback.json.gz 2026-04-04 00:46:55,358 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:46:55,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05f66d2be3e1cf2a5ae3d HTTP/1.1" 200 None 2026-04-04 00:46:55,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:46:55,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:46:55,401 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:46:55,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05f66d2be3e1cf2a5ae3d HTTP/1.1" 200 None 2026-04-04 00:46:55,442 - lava_callback - INFO - Completed processing callback for node 69d05f66d2be3e1cf2a5ae3d INFO: 172.20.0.1:55976 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:47:02,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:47:02,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 00:47:03,197 - lava_callback - INFO - Received callback for node 69d05ecbd2be3e1cf2a5aca5 INFO: 172.20.0.1:39650 - "POST /node/69d05ecbd2be3e1cf2a5aca5 HTTP/1.0" 202 Accepted 2026-04-04 00:47:03,204 - lava_callback - INFO - Starting async job submit for node 69d05ecbd2be3e1cf2a5aca5 2026-04-04 00:47:03,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:47:03,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ecbd2be3e1cf2a5aca5 HTTP/1.1" 200 None 2026-04-04 00:47:03,272 - lava_callback - DEBUG - Processing node 69d05ecbd2be3e1cf2a5aca5: baseline-arm-clabbe 2026-04-04 00:47:03,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:47:03,869 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:47:03,871 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69d05ecbd2be3e1cf2a5aca5/log.txt.gz 2026-04-04 00:47:04,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:47:04,596 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:47:04,598 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69d05ecbd2be3e1cf2a5aca5/lava-logs.yaml 2026-04-04 00:47:04,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:47:04,779 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:47:04,781 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-clabbe-69d05ecbd2be3e1cf2a5aca5/lava_callback.json.gz 2026-04-04 00:47:04,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:47:04,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ecbd2be3e1cf2a5aca5 HTTP/1.1" 200 None 2026-04-04 00:47:04,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:47:04,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:47:04,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:47:04,881 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ecbd2be3e1cf2a5aca5 HTTP/1.1" 200 None 2026-04-04 00:47:04,882 - lava_callback - INFO - Completed processing callback for node 69d05ecbd2be3e1cf2a5aca5 INFO: 172.20.0.1:39666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41720 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:47:32,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:47:32,455 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 00:47:33,878 - lava_callback - INFO - Received callback for node 69cdbc9823eb053795d42148 INFO: 172.20.0.1:54930 - "POST /node/69cdbc9823eb053795d42148 HTTP/1.0" 202 Accepted 2026-04-04 00:47:33,887 - lava_callback - INFO - Starting async job submit for node 69cdbc9823eb053795d42148 2026-04-04 00:47:33,888 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:47:33,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc9823eb053795d42148 HTTP/1.1" 200 None 2026-04-04 00:47:33,905 - lava_callback - DEBUG - Processing node 69cdbc9823eb053795d42148: baseline-arm-clabbe 2026-04-04 00:47:33,905 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbc9823eb053795d42148 2026-04-04 00:47:33,906 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:47:33,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbc9823eb053795d42148 HTTP/1.1" 200 None 2026-04-04 00:47:33,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:47:33,942 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-04 00:47:33,945 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:47:33,985 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbc9823eb053795d42148 HTTP/1.1" 200 None 2026-04-04 00:47:33,987 - lava_callback - ERROR - Error processing callback for node 69cdbc9823eb053795d42148: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38430 - "GET /node/691bc87f3e2ed70600dd0271%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38434 - "GET /node/69a161bca41e9451dcf41342%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59338 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54084 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:48:17,624 - lava_callback - INFO - Received callback for node 69d05ee3d2be3e1cf2a5ad3d INFO: 172.20.0.1:54096 - "POST /node/69d05ee3d2be3e1cf2a5ad3d HTTP/1.0" 202 Accepted 2026-04-04 00:48:17,635 - lava_callback - INFO - Starting async job submit for node 69d05ee3d2be3e1cf2a5ad3d 2026-04-04 00:48:17,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:17,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ee3d2be3e1cf2a5ad3d HTTP/1.1" 200 None 2026-04-04 00:48:17,709 - lava_callback - DEBUG - Processing node 69d05ee3d2be3e1cf2a5ad3d: ltp-containers 2026-04-04 00:48:18,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:18,655 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:18,658 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-containers-69d05ee3d2be3e1cf2a5ad3d/log.txt.gz 2026-04-04 00:48:19,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:19,322 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:19,325 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-containers-69d05ee3d2be3e1cf2a5ad3d/lava-logs.yaml 2026-04-04 00:48:19,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:19,787 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:19,789 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-containers-69d05ee3d2be3e1cf2a5ad3d/lava_callback.json.gz 2026-04-04 00:48:19,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:19,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ee3d2be3e1cf2a5ad3d HTTP/1.1" 200 None 2026-04-04 00:48:19,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:19,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:48:19,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:19,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ee3d2be3e1cf2a5ad3d HTTP/1.1" 200 None 2026-04-04 00:48:19,907 - lava_callback - INFO - Completed processing callback for node 69d05ee3d2be3e1cf2a5ad3d INFO: 172.20.0.1:56142 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:48:32,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:32,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:44330 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:48:38,463 - lava_callback - INFO - Received callback for node 69d05ee2d2be3e1cf2a5ad36 INFO: 172.20.0.1:44342 - "POST /node/69d05ee2d2be3e1cf2a5ad36 HTTP/1.0" 202 Accepted 2026-04-04 00:48:38,473 - lava_callback - INFO - Starting async job submit for node 69d05ee2d2be3e1cf2a5ad36 2026-04-04 00:48:38,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:38,531 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ee2d2be3e1cf2a5ad36 HTTP/1.1" 200 None 2026-04-04 00:48:38,532 - lava_callback - DEBUG - Processing node 69d05ee2d2be3e1cf2a5ad36: ltp-cap_bounds 2026-04-04 00:48:38,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:39,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:39,039 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69d05ee2d2be3e1cf2a5ad36/log.txt.gz 2026-04-04 00:48:39,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:39,489 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:39,491 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69d05ee2d2be3e1cf2a5ad36/lava-logs.yaml 2026-04-04 00:48:39,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:39,672 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:39,674 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cap_bounds-69d05ee2d2be3e1cf2a5ad36/lava_callback.json.gz 2026-04-04 00:48:39,699 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:39,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ee2d2be3e1cf2a5ad36 HTTP/1.1" 200 None 2026-04-04 00:48:39,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:39,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:48:39,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:39,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ee2d2be3e1cf2a5ad36 HTTP/1.1" 200 None 2026-04-04 00:48:39,764 - lava_callback - INFO - Completed processing callback for node 69d05ee2d2be3e1cf2a5ad36 2026-04-04 00:48:40,064 - lava_callback - INFO - Received callback for node 69d05ee5d2be3e1cf2a5ad40 INFO: 172.20.0.1:36022 - "POST /node/69d05ee5d2be3e1cf2a5ad40 HTTP/1.0" 202 Accepted 2026-04-04 00:48:40,074 - lava_callback - INFO - Starting async job submit for node 69d05ee5d2be3e1cf2a5ad40 2026-04-04 00:48:40,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:40,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ee5d2be3e1cf2a5ad40 HTTP/1.1" 200 None 2026-04-04 00:48:40,116 - lava_callback - DEBUG - Processing node 69d05ee5d2be3e1cf2a5ad40: ltp-controllers 2026-04-04 00:48:40,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:40,809 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:40,811 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-controllers-69d05ee5d2be3e1cf2a5ad40/log.txt.gz 2026-04-04 00:48:41,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:41,347 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:41,349 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-controllers-69d05ee5d2be3e1cf2a5ad40/lava-logs.yaml 2026-04-04 00:48:41,360 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:41,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:41,528 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-controllers-69d05ee5d2be3e1cf2a5ad40/lava_callback.json.gz 2026-04-04 00:48:41,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:41,574 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ee5d2be3e1cf2a5ad40 HTTP/1.1" 200 None 2026-04-04 00:48:41,576 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:41,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:48:41,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:41,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ee5d2be3e1cf2a5ad40 HTTP/1.1" 200 None 2026-04-04 00:48:41,628 - lava_callback - INFO - Completed processing callback for node 69d05ee5d2be3e1cf2a5ad40 2026-04-04 00:48:41,718 - lava_callback - INFO - Received callback for node 69d05ef5d2be3e1cf2a5ad99 INFO: 172.20.0.1:36036 - "POST /node/69d05ef5d2be3e1cf2a5ad99 HTTP/1.0" 202 Accepted 2026-04-04 00:48:41,731 - lava_callback - INFO - Starting async job submit for node 69d05ef5d2be3e1cf2a5ad99 2026-04-04 00:48:41,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:41,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef5d2be3e1cf2a5ad99 HTTP/1.1" 200 None 2026-04-04 00:48:41,796 - lava_callback - DEBUG - Processing node 69d05ef5d2be3e1cf2a5ad99: ltp-syscalls 2026-04-04 00:48:42,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:42,840 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:42,842 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-69d05ef5d2be3e1cf2a5ad99/log.txt.gz 2026-04-04 00:48:43,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:43,476 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:43,478 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-69d05ef5d2be3e1cf2a5ad99/lava-logs.yaml 2026-04-04 00:48:43,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:48:43,633 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:48:43,635 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-69d05ef5d2be3e1cf2a5ad99/lava_callback.json.gz 2026-04-04 00:48:43,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:43,675 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef5d2be3e1cf2a5ad99 HTTP/1.1" 200 None 2026-04-04 00:48:43,676 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:43,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:48:43,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:48:43,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ef5d2be3e1cf2a5ad99 HTTP/1.1" 200 None 2026-04-04 00:48:43,725 - lava_callback - INFO - Completed processing callback for node 69d05ef5d2be3e1cf2a5ad99 INFO: 172.20.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39894 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:49:02,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:49:02,505 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52820 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:49:38,166 - lava_callback - INFO - Received callback for node 69d05eead2be3e1cf2a5ad49 INFO: 172.20.0.1:52828 - "POST /node/69d05eead2be3e1cf2a5ad49 HTTP/1.0" 202 Accepted 2026-04-04 00:49:38,173 - lava_callback - INFO - Starting async job submit for node 69d05eead2be3e1cf2a5ad49 2026-04-04 00:49:38,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:49:38,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eead2be3e1cf2a5ad49 HTTP/1.1" 200 None 2026-04-04 00:49:38,236 - lava_callback - DEBUG - Processing node 69d05eead2be3e1cf2a5ad49: ltp-fs_bind 2026-04-04 00:49:38,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:49:39,052 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:49:39,054 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69d05eead2be3e1cf2a5ad49/log.txt.gz 2026-04-04 00:49:39,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:49:39,662 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:49:39,664 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69d05eead2be3e1cf2a5ad49/lava-logs.yaml 2026-04-04 00:49:39,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:49:39,983 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:49:39,985 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fs_bind-69d05eead2be3e1cf2a5ad49/lava_callback.json.gz 2026-04-04 00:49:40,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:49:40,065 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eead2be3e1cf2a5ad49 HTTP/1.1" 200 None 2026-04-04 00:49:40,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:49:40,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:49:40,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:49:40,132 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05eead2be3e1cf2a5ad49 HTTP/1.1" 200 None 2026-04-04 00:49:40,134 - lava_callback - INFO - Completed processing callback for node 69d05eead2be3e1cf2a5ad49 INFO: 172.20.0.1:39796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55198 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:50:02,507 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:50:02,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36254 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49590 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:50:38,242 - lava_callback - INFO - Received callback for node 69d05ee8d2be3e1cf2a5ad43 INFO: 172.20.0.1:49592 - "POST /node/69d05ee8d2be3e1cf2a5ad43 HTTP/1.0" 202 Accepted 2026-04-04 00:50:38,256 - lava_callback - INFO - Starting async job submit for node 69d05ee8d2be3e1cf2a5ad43 2026-04-04 00:50:38,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:50:38,331 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ee8d2be3e1cf2a5ad43 HTTP/1.1" 200 None 2026-04-04 00:50:38,332 - lava_callback - DEBUG - Processing node 69d05ee8d2be3e1cf2a5ad43: ltp-cve 2026-04-04 00:50:38,943 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:50:39,552 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:50:39,554 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-cve-69d05ee8d2be3e1cf2a5ad43/log.txt.gz 2026-04-04 00:50:39,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:50:40,821 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:50:40,823 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-cve-69d05ee8d2be3e1cf2a5ad43/lava-logs.yaml 2026-04-04 00:50:40,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:50:40,989 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:50:40,991 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-cve-69d05ee8d2be3e1cf2a5ad43/lava_callback.json.gz 2026-04-04 00:50:41,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:50:41,032 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ee8d2be3e1cf2a5ad43 HTTP/1.1" 200 None 2026-04-04 00:50:41,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:50:41,048 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:50:41,050 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:50:41,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ee8d2be3e1cf2a5ad43 HTTP/1.1" 200 None 2026-04-04 00:50:41,089 - lava_callback - INFO - Completed processing callback for node 69d05ee8d2be3e1cf2a5ad43 INFO: 172.20.0.1:58940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58480 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:51:02,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:51:02,557 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48688 - "GET /node/694da1289de8dfc8c2ed56ba%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48694 - "GET /node/699363a8cebe99dd8e063f83%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:48710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38266 - "GET /node/694b025e6c96ed51d3173101%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38278 - "GET /node/698771f4e31b58f834f1a7be%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-04 00:51:54,150 - lava_callback - INFO - Received callback for node 69cdbd9b23eb053795d421fb INFO: 172.20.0.1:38288 - "POST /node/69cdbd9b23eb053795d421fb HTTP/1.0" 202 Accepted 2026-04-04 00:51:54,158 - lava_callback - INFO - Starting async job submit for node 69cdbd9b23eb053795d421fb 2026-04-04 00:51:54,160 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:51:54,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbd9b23eb053795d421fb HTTP/1.1" 200 None 2026-04-04 00:51:54,191 - lava_callback - DEBUG - Processing node 69cdbd9b23eb053795d421fb: baseline-arm-clabbe 2026-04-04 00:51:54,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbd9b23eb053795d421fb 2026-04-04 00:51:54,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:51:54,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbd9b23eb053795d421fb HTTP/1.1" 200 None 2026-04-04 00:51:54,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:51:54,241 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 00:51:54,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:51:54,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbd9b23eb053795d421fb HTTP/1.1" 200 None 2026-04-04 00:51:54,286 - lava_callback - ERROR - Error processing callback for node 69cdbd9b23eb053795d421fb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55090 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55106 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:52:17,444 - lava_callback - INFO - Received callback for node 69d05eecd2be3e1cf2a5ad7c INFO: 172.20.0.1:55112 - "POST /node/69d05eecd2be3e1cf2a5ad7c HTTP/1.0" 202 Accepted 2026-04-04 00:52:17,455 - lava_callback - INFO - Starting async job submit for node 69d05eecd2be3e1cf2a5ad7c 2026-04-04 00:52:17,506 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:52:17,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eecd2be3e1cf2a5ad7c HTTP/1.1" 200 None 2026-04-04 00:52:17,534 - lava_callback - DEBUG - Processing node 69d05eecd2be3e1cf2a5ad7c: ltp-fsx 2026-04-04 00:52:18,179 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:52:19,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:52:19,203 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-fsx-69d05eecd2be3e1cf2a5ad7c/log.txt.gz 2026-04-04 00:52:19,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:52:21,526 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:52:21,528 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-fsx-69d05eecd2be3e1cf2a5ad7c/lava-logs.yaml 2026-04-04 00:52:21,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:52:22,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:52:22,596 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-fsx-69d05eecd2be3e1cf2a5ad7c/lava_callback.json.gz 2026-04-04 00:52:22,717 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:52:22,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eecd2be3e1cf2a5ad7c HTTP/1.1" 200 None 2026-04-04 00:52:22,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:52:22,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:52:22,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:52:22,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05eecd2be3e1cf2a5ad7c HTTP/1.1" 200 None 2026-04-04 00:52:22,839 - lava_callback - INFO - Completed processing callback for node 69d05eecd2be3e1cf2a5ad7c INFO: 172.20.0.1:58654 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:52:32,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:52:32,588 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35610 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:52:38,264 - lava_callback - INFO - Received callback for node 69d05eedd2be3e1cf2a5ad80 INFO: 172.20.0.1:35626 - "POST /node/69d05eedd2be3e1cf2a5ad80 HTTP/1.0" 202 Accepted 2026-04-04 00:52:38,275 - lava_callback - INFO - Starting async job submit for node 69d05eedd2be3e1cf2a5ad80 2026-04-04 00:52:38,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:52:38,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eedd2be3e1cf2a5ad80 HTTP/1.1" 200 None 2026-04-04 00:52:38,351 - lava_callback - DEBUG - Processing node 69d05eedd2be3e1cf2a5ad80: ltp-input 2026-04-04 00:52:38,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:52:40,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:52:40,062 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-input-69d05eedd2be3e1cf2a5ad80/log.txt.gz 2026-04-04 00:52:40,470 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:52:42,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:52:42,323 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-input-69d05eedd2be3e1cf2a5ad80/lava-logs.yaml 2026-04-04 00:52:42,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:52:43,506 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:52:43,509 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-input-69d05eedd2be3e1cf2a5ad80/lava_callback.json.gz 2026-04-04 00:52:43,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:52:43,595 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eedd2be3e1cf2a5ad80 HTTP/1.1" 200 None 2026-04-04 00:52:43,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:52:43,628 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:52:43,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:52:43,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05eedd2be3e1cf2a5ad80 HTTP/1.1" 200 None 2026-04-04 00:52:43,674 - lava_callback - INFO - Completed processing callback for node 69d05eedd2be3e1cf2a5ad80 INFO: 172.20.0.1:50844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41610 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:53:02,590 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:02,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:57032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57038 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:53:14,158 - lava_callback - INFO - Received callback for node 69cdbde523eb053795d42222 INFO: 172.20.0.1:41950 - "POST /node/69cdbde523eb053795d42222 HTTP/1.0" 202 Accepted 2026-04-04 00:53:14,167 - lava_callback - INFO - Starting async job submit for node 69cdbde523eb053795d42222 2026-04-04 00:53:14,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:14,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbde523eb053795d42222 HTTP/1.1" 200 None 2026-04-04 00:53:14,198 - lava_callback - DEBUG - Processing node 69cdbde523eb053795d42222: baseline-arm-clabbe 2026-04-04 00:53:14,198 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbde523eb053795d42222 2026-04-04 00:53:14,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:14,221 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbde523eb053795d42222 HTTP/1.1" 200 None 2026-04-04 00:53:14,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:14,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 00:53:14,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:14,298 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbde523eb053795d42222 HTTP/1.1" 200 None 2026-04-04 00:53:14,300 - lava_callback - ERROR - Error processing callback for node 69cdbde523eb053795d42222: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41952 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41962 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:53:17,467 - lava_callback - INFO - Received callback for node 69d05c80d2be3e1cf2a5abc8 INFO: 172.20.0.1:41974 - "POST /node/69d05c80d2be3e1cf2a5abc8 HTTP/1.0" 202 Accepted 2026-04-04 00:53:17,478 - lava_callback - INFO - Starting async job submit for node 69d05c80d2be3e1cf2a5abc8 2026-04-04 00:53:17,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:17,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05c80d2be3e1cf2a5abc8 HTTP/1.1" 200 None 2026-04-04 00:53:17,618 - lava_callback - DEBUG - Processing node 69d05c80d2be3e1cf2a5abc8: ltp-smoke 2026-04-04 00:53:18,276 - lava_callback - INFO - Received callback for node 69d05eefd2be3e1cf2a5ad83 INFO: 172.20.0.1:41978 - "POST /node/69d05eefd2be3e1cf2a5ad83 HTTP/1.0" 202 Accepted 2026-04-04 00:53:18,305 - lava_callback - INFO - Starting async job submit for node 69d05eefd2be3e1cf2a5ad83 2026-04-04 00:53:18,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:18,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eefd2be3e1cf2a5ad83 HTTP/1.1" 200 None 2026-04-04 00:53:18,483 - lava_callback - DEBUG - Processing node 69d05eefd2be3e1cf2a5ad83: ltp-ipc 2026-04-04 00:53:18,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:53:19,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:53:19,508 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:53:19,510 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69d05c80d2be3e1cf2a5abc8/log.txt.gz 2026-04-04 00:53:20,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:53:20,595 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:53:20,598 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-ipc-69d05eefd2be3e1cf2a5ad83/log.txt.gz 2026-04-04 00:53:21,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:53:21,962 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:53:21,964 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69d05c80d2be3e1cf2a5abc8/lava-logs.yaml 2026-04-04 00:53:21,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:53:22,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:53:22,726 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-ipc-69d05eefd2be3e1cf2a5ad83/lava-logs.yaml 2026-04-04 00:53:22,756 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:53:23,221 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:53:23,223 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69d05c80d2be3e1cf2a5abc8/lava_callback.json.gz 2026-04-04 00:53:23,444 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:23,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05c80d2be3e1cf2a5abc8 HTTP/1.1" 200 None 2026-04-04 00:53:23,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:23,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cfd2be3e1cf2a5a7f5 HTTP/1.1" 200 None 2026-04-04 00:53:23,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:23,571 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05c80d2be3e1cf2a5abc8 HTTP/1.1" 200 None 2026-04-04 00:53:23,782 - lava_callback - INFO - Completed processing callback for node 69d05c80d2be3e1cf2a5abc8 2026-04-04 00:53:24,193 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:53:24,196 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-ipc-69d05eefd2be3e1cf2a5ad83/lava_callback.json.gz 2026-04-04 00:53:24,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:24,276 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eefd2be3e1cf2a5ad83 HTTP/1.1" 200 None 2026-04-04 00:53:24,279 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:24,307 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:53:24,309 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:24,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05eefd2be3e1cf2a5ad83 HTTP/1.1" 200 None 2026-04-04 00:53:24,355 - lava_callback - INFO - Completed processing callback for node 69d05eefd2be3e1cf2a5ad83 INFO: 172.20.0.1:57216 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:53:32,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:32,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41502 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:53:38,645 - lava_callback - INFO - Received callback for node 69d05ef1d2be3e1cf2a5ad8c INFO: 172.20.0.1:41508 - "POST /node/69d05ef1d2be3e1cf2a5ad8c HTTP/1.0" 202 Accepted 2026-04-04 00:53:38,656 - lava_callback - INFO - Starting async job submit for node 69d05ef1d2be3e1cf2a5ad8c 2026-04-04 00:53:38,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:38,729 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef1d2be3e1cf2a5ad8c HTTP/1.1" 200 None 2026-04-04 00:53:38,731 - lava_callback - DEBUG - Processing node 69d05ef1d2be3e1cf2a5ad8c: ltp-pty 2026-04-04 00:53:39,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:53:39,958 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:53:39,960 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-pty-69d05ef1d2be3e1cf2a5ad8c/log.txt.gz 2026-04-04 00:53:40,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:53:41,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:53:41,784 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-pty-69d05ef1d2be3e1cf2a5ad8c/lava-logs.yaml 2026-04-04 00:53:41,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:53:43,162 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:53:43,164 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-pty-69d05ef1d2be3e1cf2a5ad8c/lava_callback.json.gz 2026-04-04 00:53:43,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:43,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef1d2be3e1cf2a5ad8c HTTP/1.1" 200 None 2026-04-04 00:53:43,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:43,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:53:43,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:43,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ef1d2be3e1cf2a5ad8c HTTP/1.1" 200 None 2026-04-04 00:53:43,338 - lava_callback - INFO - Completed processing callback for node 69d05ef1d2be3e1cf2a5ad8c INFO: 172.20.0.1:53244 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:53:54,184 - lava_callback - INFO - Received callback for node 69cdbe0c23eb053795d42231 INFO: 172.20.0.1:44556 - "POST /node/69cdbe0c23eb053795d42231 HTTP/1.0" 202 Accepted 2026-04-04 00:53:54,193 - lava_callback - INFO - Starting async job submit for node 69cdbe0c23eb053795d42231 2026-04-04 00:53:54,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:54,224 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbe0c23eb053795d42231 HTTP/1.1" 200 None 2026-04-04 00:53:54,226 - lava_callback - DEBUG - Processing node 69cdbe0c23eb053795d42231: baseline-arm-clabbe 2026-04-04 00:53:54,226 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbe0c23eb053795d42231 2026-04-04 00:53:54,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:54,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbe0c23eb053795d42231 HTTP/1.1" 200 None 2026-04-04 00:53:54,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:54,278 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 00:53:54,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:53:54,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbe0c23eb053795d42231 HTTP/1.1" 200 None 2026-04-04 00:53:54,318 - lava_callback - ERROR - Error processing callback for node 69cdbe0c23eb053795d42231: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44568 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:54:02,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:54:02,705 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49248 - "GET /node/693b19cb15f407b7629df7b2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:43368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38560 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59668 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51280 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:56:14,347 - lava_callback - INFO - Received callback for node 69cdbea123eb053795d42275 INFO: 172.20.0.1:55862 - "POST /node/69cdbea123eb053795d42275 HTTP/1.0" 202 Accepted 2026-04-04 00:56:14,354 - lava_callback - INFO - Starting async job submit for node 69cdbea123eb053795d42275 2026-04-04 00:56:14,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:14,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbea123eb053795d42275 HTTP/1.1" 200 None 2026-04-04 00:56:14,372 - lava_callback - DEBUG - Processing node 69cdbea123eb053795d42275: baseline-arm-clabbe 2026-04-04 00:56:14,372 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbea123eb053795d42275 2026-04-04 00:56:14,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:14,387 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbea123eb053795d42275 HTTP/1.1" 200 None 2026-04-04 00:56:14,388 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:14,402 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 00:56:14,403 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:14,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbea123eb053795d42275 HTTP/1.1" 200 None 2026-04-04 00:56:14,432 - lava_callback - ERROR - Error processing callback for node 69cdbea123eb053795d42275: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55870 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55876 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:56:16,874 - lava_callback - INFO - Received callback for node 69d0618bd2be3e1cf2a5b06e INFO: 172.20.0.1:55886 - "POST /node/69d0618bd2be3e1cf2a5b06e HTTP/1.0" 202 Accepted 2026-04-04 00:56:16,880 - lava_callback - INFO - Starting async job submit for node 69d0618bd2be3e1cf2a5b06e 2026-04-04 00:56:16,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:16,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618bd2be3e1cf2a5b06e HTTP/1.1" 200 None 2026-04-04 00:56:16,914 - lava_callback - DEBUG - Processing node 69d0618bd2be3e1cf2a5b06e: baseline-arm64 2026-04-04 00:56:17,104 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:17,226 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:17,227 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618bd2be3e1cf2a5b06e/log.txt.gz 2026-04-04 00:56:17,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:17,547 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:17,549 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618bd2be3e1cf2a5b06e/lava-logs.yaml 2026-04-04 00:56:17,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:17,573 - lava_callback - INFO - Received callback for node 69d05ef4d2be3e1cf2a5ad92 INFO: 172.20.0.1:55898 - "POST /node/69d05ef4d2be3e1cf2a5ad92 HTTP/1.0" 202 Accepted 2026-04-04 00:56:17,583 - lava_callback - INFO - Starting async job submit for node 69d05ef4d2be3e1cf2a5ad92 2026-04-04 00:56:17,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:17,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef4d2be3e1cf2a5ad92 HTTP/1.1" 200 None 2026-04-04 00:56:17,659 - lava_callback - DEBUG - Processing node 69d05ef4d2be3e1cf2a5ad92: ltp-smoke 2026-04-04 00:56:17,791 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:17,824 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618bd2be3e1cf2a5b06e/lava_callback.json.gz 2026-04-04 00:56:17,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:17,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618bd2be3e1cf2a5b06e HTTP/1.1" 200 None 2026-04-04 00:56:17,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:18,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:56:18,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:18,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:18,225 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0618bd2be3e1cf2a5b06e HTTP/1.1" 200 None 2026-04-04 00:56:18,226 - lava_callback - INFO - Completed processing callback for node 69d0618bd2be3e1cf2a5b06e 2026-04-04 00:56:18,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:18,463 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69d05ef4d2be3e1cf2a5ad92/log.txt.gz 2026-04-04 00:56:18,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:18,961 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:18,964 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69d05ef4d2be3e1cf2a5ad92/lava-logs.yaml 2026-04-04 00:56:18,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:19,181 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:19,183 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69d05ef4d2be3e1cf2a5ad92/lava_callback.json.gz 2026-04-04 00:56:19,210 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:19,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef4d2be3e1cf2a5ad92 HTTP/1.1" 200 None 2026-04-04 00:56:19,228 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:19,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:56:19,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:19,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ef4d2be3e1cf2a5ad92 HTTP/1.1" 200 None 2026-04-04 00:56:19,285 - lava_callback - INFO - Completed processing callback for node 69d05ef4d2be3e1cf2a5ad92 INFO: 172.20.0.1:39854 - "GET /node/693e143a407f32424eb56a54%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39870 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:56:32,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:32,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 00:56:35,577 - lava_callback - INFO - Received callback for node 69d06197d2be3e1cf2a5b08f INFO: 172.20.0.1:54506 - "POST /node/69d06197d2be3e1cf2a5b08f HTTP/1.0" 202 Accepted 2026-04-04 00:56:35,584 - lava_callback - INFO - Starting async job submit for node 69d06197d2be3e1cf2a5b08f 2026-04-04 00:56:35,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:35,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06197d2be3e1cf2a5b08f HTTP/1.1" 200 None 2026-04-04 00:56:35,615 - lava_callback - DEBUG - Processing node 69d06197d2be3e1cf2a5b08f: baseline-arm64 2026-04-04 00:56:35,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:35,896 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:35,899 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06197d2be3e1cf2a5b08f/log.txt.gz 2026-04-04 00:56:36,027 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:36,234 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:36,236 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06197d2be3e1cf2a5b08f/lava-logs.yaml 2026-04-04 00:56:36,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:36,338 - lava_callback - INFO - Received callback for node 69d061a7d2be3e1cf2a5b0e5 INFO: 172.20.0.1:54518 - "POST /node/69d061a7d2be3e1cf2a5b0e5 HTTP/1.0" 202 Accepted 2026-04-04 00:56:36,349 - lava_callback - INFO - Starting async job submit for node 69d061a7d2be3e1cf2a5b0e5 2026-04-04 00:56:36,351 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:36,366 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a7d2be3e1cf2a5b0e5 HTTP/1.1" 200 None 2026-04-04 00:56:36,367 - lava_callback - DEBUG - Processing node 69d061a7d2be3e1cf2a5b0e5: baseline-nfs-arm64 2026-04-04 00:56:36,368 - lava_callback - WARNING - Failed to upload log 2026-04-04 00:56:36,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:36,416 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:36,418 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06197d2be3e1cf2a5b08f/lava_callback.json.gz 2026-04-04 00:56:36,442 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:36,462 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06197d2be3e1cf2a5b08f HTTP/1.1" 200 None 2026-04-04 00:56:36,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:36,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:56:36,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:36,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06197d2be3e1cf2a5b08f HTTP/1.1" 200 None 2026-04-04 00:56:36,514 - lava_callback - INFO - Completed processing callback for node 69d06197d2be3e1cf2a5b08f INFO: 172.20.0.1:54524 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:56:36,562 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:36,564 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a7d2be3e1cf2a5b0e5/lava-logs.yaml 2026-04-04 00:56:36,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:36,730 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:36,732 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a7d2be3e1cf2a5b0e5/lava_callback.json.gz 2026-04-04 00:56:36,735 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:36,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a7d2be3e1cf2a5b0e5 HTTP/1.1" 200 None 2026-04-04 00:56:36,764 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:36,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:56:36,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:36,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061a7d2be3e1cf2a5b0e5 HTTP/1.1" 200 None 2026-04-04 00:56:36,859 - lava_callback - INFO - Completed processing callback for node 69d061a7d2be3e1cf2a5b0e5 2026-04-04 00:56:38,860 - lava_callback - INFO - Received callback for node 69d05ef7d2be3e1cf2a5ad9c INFO: 172.20.0.1:54528 - "POST /node/69d05ef7d2be3e1cf2a5ad9c HTTP/1.0" 202 Accepted 2026-04-04 00:56:38,866 - lava_callback - INFO - Starting async job submit for node 69d05ef7d2be3e1cf2a5ad9c 2026-04-04 00:56:38,895 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:38,913 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef7d2be3e1cf2a5ad9c HTTP/1.1" 200 None 2026-04-04 00:56:38,914 - lava_callback - DEBUG - Processing node 69d05ef7d2be3e1cf2a5ad9c: ltp-syscalls-ipc 2026-04-04 00:56:39,464 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:39,617 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:39,619 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69d05ef7d2be3e1cf2a5ad9c/log.txt.gz 2026-04-04 00:56:39,927 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:40,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:40,245 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69d05ef7d2be3e1cf2a5ad9c/lava-logs.yaml 2026-04-04 00:56:40,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:40,461 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:40,463 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-syscalls-ipc-69d05ef7d2be3e1cf2a5ad9c/lava_callback.json.gz 2026-04-04 00:56:40,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:40,530 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef7d2be3e1cf2a5ad9c HTTP/1.1" 200 None 2026-04-04 00:56:40,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:40,556 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:56:40,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:40,616 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ef7d2be3e1cf2a5ad9c HTTP/1.1" 200 None 2026-04-04 00:56:40,617 - lava_callback - INFO - Completed processing callback for node 69d05ef7d2be3e1cf2a5ad9c 2026-04-04 00:56:42,735 - lava_callback - INFO - Received callback for node 69d0618cd2be3e1cf2a5b074 INFO: 172.20.0.1:37680 - "POST /node/69d0618cd2be3e1cf2a5b074 HTTP/1.0" 202 Accepted 2026-04-04 00:56:42,744 - lava_callback - INFO - Starting async job submit for node 69d0618cd2be3e1cf2a5b074 2026-04-04 00:56:42,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:42,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618cd2be3e1cf2a5b074 HTTP/1.1" 200 None 2026-04-04 00:56:42,788 - lava_callback - DEBUG - Processing node 69d0618cd2be3e1cf2a5b074: baseline-arm64 2026-04-04 00:56:43,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:43,428 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:43,430 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618cd2be3e1cf2a5b074/log.txt.gz 2026-04-04 00:56:43,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:43,932 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:43,934 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618cd2be3e1cf2a5b074/lava-logs.yaml 2026-04-04 00:56:43,942 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:56:44,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:56:44,097 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618cd2be3e1cf2a5b074/lava_callback.json.gz 2026-04-04 00:56:44,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:44,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618cd2be3e1cf2a5b074 HTTP/1.1" 200 None 2026-04-04 00:56:44,166 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:44,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:56:44,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:56:44,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0618cd2be3e1cf2a5b074 HTTP/1.1" 200 None 2026-04-04 00:56:44,237 - lava_callback - INFO - Completed processing callback for node 69d0618cd2be3e1cf2a5b074 INFO: 172.20.0.1:37684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43238 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:57:02,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:02,750 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:35812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35826 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:57:07,534 - lava_callback - INFO - Received callback for node 69d0618bd2be3e1cf2a5b071 INFO: 172.20.0.1:35832 - "POST /node/69d0618bd2be3e1cf2a5b071 HTTP/1.0" 202 Accepted 2026-04-04 00:57:07,543 - lava_callback - INFO - Starting async job submit for node 69d0618bd2be3e1cf2a5b071 2026-04-04 00:57:07,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:07,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618bd2be3e1cf2a5b071 HTTP/1.1" 200 None 2026-04-04 00:57:07,593 - lava_callback - DEBUG - Processing node 69d0618bd2be3e1cf2a5b071: baseline-arm64 2026-04-04 00:57:07,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:08,042 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:08,044 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618bd2be3e1cf2a5b071/log.txt.gz 2026-04-04 00:57:08,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:08,409 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:08,411 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618bd2be3e1cf2a5b071/lava-logs.yaml 2026-04-04 00:57:08,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:08,561 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:08,563 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618bd2be3e1cf2a5b071/lava_callback.json.gz 2026-04-04 00:57:08,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:08,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618bd2be3e1cf2a5b071 HTTP/1.1" 200 None 2026-04-04 00:57:08,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:08,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:57:08,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:08,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0618bd2be3e1cf2a5b071 HTTP/1.1" 200 None 2026-04-04 00:57:08,695 - lava_callback - INFO - Completed processing callback for node 69d0618bd2be3e1cf2a5b071 INFO: 172.20.0.1:41592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43676 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:57:32,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:32,774 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 00:57:34,474 - lava_callback - INFO - Received callback for node 69cdbeea23eb053795d422a7 INFO: 172.20.0.1:37400 - "POST /node/69cdbeea23eb053795d422a7 HTTP/1.0" 202 Accepted 2026-04-04 00:57:34,481 - lava_callback - INFO - Starting async job submit for node 69cdbeea23eb053795d422a7 2026-04-04 00:57:34,482 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:34,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbeea23eb053795d422a7 HTTP/1.1" 200 None 2026-04-04 00:57:34,498 - lava_callback - DEBUG - Processing node 69cdbeea23eb053795d422a7: baseline-arm-clabbe 2026-04-04 00:57:34,499 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbeea23eb053795d422a7 2026-04-04 00:57:34,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:34,514 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbeea23eb053795d422a7 HTTP/1.1" 200 None 2026-04-04 00:57:34,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:34,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 00:57:34,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:34,567 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbeea23eb053795d422a7 HTTP/1.1" 200 None 2026-04-04 00:57:34,569 - lava_callback - ERROR - Error processing callback for node 69cdbeea23eb053795d422a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37408 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:57:38,495 - lava_callback - INFO - Received callback for node 69d05efad2be3e1cf2a5ada2 INFO: 172.20.0.1:37414 - "POST /node/69d05efad2be3e1cf2a5ada2 HTTP/1.0" 202 Accepted 2026-04-04 00:57:38,505 - lava_callback - INFO - Starting async job submit for node 69d05efad2be3e1cf2a5ada2 2026-04-04 00:57:38,532 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:38,547 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05efad2be3e1cf2a5ada2 HTTP/1.1" 200 None 2026-04-04 00:57:38,548 - lava_callback - DEBUG - Processing node 69d05efad2be3e1cf2a5ada2: ltp-watchqueue 2026-04-04 00:57:38,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:39,121 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:39,123 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69d05efad2be3e1cf2a5ada2/log.txt.gz 2026-04-04 00:57:39,379 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:39,587 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:39,589 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69d05efad2be3e1cf2a5ada2/lava-logs.yaml 2026-04-04 00:57:39,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:39,768 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:39,771 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-watchqueue-69d05efad2be3e1cf2a5ada2/lava_callback.json.gz 2026-04-04 00:57:39,797 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:39,811 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05efad2be3e1cf2a5ada2 HTTP/1.1" 200 None 2026-04-04 00:57:39,813 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:39,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:57:39,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:39,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05efad2be3e1cf2a5ada2 HTTP/1.1" 200 None 2026-04-04 00:57:39,858 - lava_callback - INFO - Completed processing callback for node 69d05efad2be3e1cf2a5ada2 2026-04-04 00:57:46,027 - lava_callback - INFO - Received callback for node 69d061d2d2be3e1cf2a5b197 INFO: 172.20.0.1:51584 - "POST /node/69d061d2d2be3e1cf2a5b197 HTTP/1.0" 202 Accepted 2026-04-04 00:57:46,035 - lava_callback - INFO - Starting async job submit for node 69d061d2d2be3e1cf2a5b197 2026-04-04 00:57:46,071 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:46,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061d2d2be3e1cf2a5b197 HTTP/1.1" 200 None 2026-04-04 00:57:46,101 - lava_callback - DEBUG - Processing node 69d061d2d2be3e1cf2a5b197: baseline-arm64-foundriesio 2026-04-04 00:57:46,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:51600 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:57:46,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:46,527 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d061d2d2be3e1cf2a5b197/log.txt.gz 2026-04-04 00:57:46,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:46,899 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:46,901 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d061d2d2be3e1cf2a5b197/lava-logs.yaml 2026-04-04 00:57:46,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:47,066 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:47,068 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d061d2d2be3e1cf2a5b197/lava_callback.json.gz 2026-04-04 00:57:47,097 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:47,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061d2d2be3e1cf2a5b197 HTTP/1.1" 200 None 2026-04-04 00:57:47,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:47,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:57:47,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:47,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061d2d2be3e1cf2a5b197 HTTP/1.1" 200 None 2026-04-04 00:57:47,181 - lava_callback - INFO - Completed processing callback for node 69d061d2d2be3e1cf2a5b197 2026-04-04 00:57:55,599 - lava_callback - INFO - Received callback for node 69d061d0d2be3e1cf2a5b194 INFO: 172.20.0.1:55296 - "POST /node/69d061d0d2be3e1cf2a5b194 HTTP/1.0" 202 Accepted 2026-04-04 00:57:55,606 - lava_callback - INFO - Starting async job submit for node 69d061d0d2be3e1cf2a5b194 2026-04-04 00:57:55,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:55,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061d0d2be3e1cf2a5b194 HTTP/1.1" 200 None 2026-04-04 00:57:55,659 - lava_callback - DEBUG - Processing node 69d061d0d2be3e1cf2a5b194: baseline-arm64-pengutronix 2026-04-04 00:57:55,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:56,059 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:56,060 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d061d0d2be3e1cf2a5b194/log.txt.gz 2026-04-04 00:57:56,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:56,410 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:56,411 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d061d0d2be3e1cf2a5b194/lava-logs.yaml 2026-04-04 00:57:56,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:55304 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:57:56,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:56,572 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d061d0d2be3e1cf2a5b194/lava_callback.json.gz 2026-04-04 00:57:56,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:56,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061d0d2be3e1cf2a5b194 HTTP/1.1" 200 None 2026-04-04 00:57:56,609 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:56,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:57:56,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:56,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061d0d2be3e1cf2a5b194 HTTP/1.1" 200 None 2026-04-04 00:57:56,651 - lava_callback - INFO - Completed processing callback for node 69d061d0d2be3e1cf2a5b194 2026-04-04 00:57:57,781 - lava_callback - INFO - Received callback for node 69d061b8d2be3e1cf2a5b114 INFO: 172.20.0.1:55308 - "POST /node/69d061b8d2be3e1cf2a5b114 HTTP/1.0" 202 Accepted 2026-04-04 00:57:57,788 - lava_callback - INFO - Starting async job submit for node 69d061b8d2be3e1cf2a5b114 2026-04-04 00:57:57,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:57,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061b8d2be3e1cf2a5b114 HTTP/1.1" 200 None 2026-04-04 00:57:57,839 - lava_callback - DEBUG - Processing node 69d061b8d2be3e1cf2a5b114: baseline-arm64-broonie 2026-04-04 00:57:58,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:58,200 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:58,202 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061b8d2be3e1cf2a5b114/log.txt.gz INFO: 172.20.0.1:55322 - "GET /node/69b13cb1a870a1dc3343c6b2%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-04 00:57:58,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:58,439 - lava_callback - INFO - Received callback for node 69d05ef8d2be3e1cf2a5ad9f INFO: 172.20.0.1:55334 - "POST /node/69d05ef8d2be3e1cf2a5ad9f HTTP/1.0" 202 Accepted 2026-04-04 00:57:58,448 - lava_callback - INFO - Starting async job submit for node 69d05ef8d2be3e1cf2a5ad9f 2026-04-04 00:57:58,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:58,497 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef8d2be3e1cf2a5ad9f HTTP/1.1" 200 None 2026-04-04 00:57:58,498 - lava_callback - DEBUG - Processing node 69d05ef8d2be3e1cf2a5ad9f: ltp-timers 2026-04-04 00:57:58,593 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:58,605 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061b8d2be3e1cf2a5b114/lava-logs.yaml 2026-04-04 00:57:58,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:58,853 - lava_callback - INFO - Received callback for node 69d061ccd2be3e1cf2a5b18b INFO: 172.20.0.1:55348 - "POST /node/69d061ccd2be3e1cf2a5b18b HTTP/1.0" 202 Accepted 2026-04-04 00:57:58,878 - lava_callback - INFO - Starting async job submit for node 69d061ccd2be3e1cf2a5b18b 2026-04-04 00:57:58,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:58,970 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:59,018 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061b8d2be3e1cf2a5b114/lava_callback.json.gz 2026-04-04 00:57:59,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:59,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061ccd2be3e1cf2a5b18b HTTP/1.1" 200 None 2026-04-04 00:57:59,079 - lava_callback - DEBUG - Processing node 69d061ccd2be3e1cf2a5b18b: baseline-arm64-kontron 2026-04-04 00:57:59,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:59,156 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061b8d2be3e1cf2a5b114 HTTP/1.1" 200 None 2026-04-04 00:57:59,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:59,246 - lava_callback - INFO - Received callback for node 69d061c0d2be3e1cf2a5b154 2026-04-04 00:57:59,262 - lava_callback - INFO - Starting async job submit for node 69d061c0d2be3e1cf2a5b154 INFO: 172.20.0.1:55350 - "POST /node/69d061c0d2be3e1cf2a5b154 HTTP/1.0" 202 Accepted 2026-04-04 00:57:59,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:59,364 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:57:59,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:57:59,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:57:59,389 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-timers-69d05ef8d2be3e1cf2a5ad9f/log.txt.gz 2026-04-04 00:57:59,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061c0d2be3e1cf2a5b154 HTTP/1.1" 200 None 2026-04-04 00:57:59,460 - lava_callback - DEBUG - Processing node 69d061c0d2be3e1cf2a5b154: baseline-arm64-broonie 2026-04-04 00:57:59,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061b8d2be3e1cf2a5b114 HTTP/1.1" 200 None 2026-04-04 00:57:59,486 - lava_callback - INFO - Completed processing callback for node 69d061b8d2be3e1cf2a5b114 2026-04-04 00:57:59,533 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:57:59,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:00,009 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d061ccd2be3e1cf2a5b18b/log.txt.gz 2026-04-04 00:58:00,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:00,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:00,152 - lava_callback - INFO - Received callback for node 69d061ced2be3e1cf2a5b18e INFO: 172.20.0.1:36990 - "POST /node/69d061ced2be3e1cf2a5b18e HTTP/1.0" 202 Accepted 2026-04-04 00:58:00,166 - lava_callback - INFO - Starting async job submit for node 69d061ced2be3e1cf2a5b18e 2026-04-04 00:58:00,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:00,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:00,214 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061ced2be3e1cf2a5b18e HTTP/1.1" 200 None 2026-04-04 00:58:00,215 - lava_callback - DEBUG - Processing node 69d061ced2be3e1cf2a5b18e: baseline-arm64-kontron 2026-04-04 00:58:00,312 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:00,360 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c0d2be3e1cf2a5b154/log.txt.gz 2026-04-04 00:58:00,431 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:00,467 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:00,539 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-timers-69d05ef8d2be3e1cf2a5ad9f/lava-logs.yaml 2026-04-04 00:58:00,540 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d061ccd2be3e1cf2a5b18b/lava-logs.yaml 2026-04-04 00:58:00,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:00,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:00,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:00,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:00,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:00,899 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d061ccd2be3e1cf2a5b18b/lava_callback.json.gz 2026-04-04 00:58:00,923 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:00,930 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:00,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:00,935 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d061ced2be3e1cf2a5b18e/log.txt.gz 2026-04-04 00:58:00,935 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-timers-69d05ef8d2be3e1cf2a5ad9f/lava_callback.json.gz 2026-04-04 00:58:01,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:01,045 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:01,071 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c0d2be3e1cf2a5b154/lava-logs.yaml 2026-04-04 00:58:01,080 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061ccd2be3e1cf2a5b18b HTTP/1.1" 200 None 2026-04-04 00:58:01,098 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef8d2be3e1cf2a5ad9f HTTP/1.1" 200 None 2026-04-04 00:58:01,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:01,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:01,177 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:01,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:58:01,193 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:58:01,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:01,197 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:01,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:01,238 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ef8d2be3e1cf2a5ad9f HTTP/1.1" 200 None 2026-04-04 00:58:01,240 - lava_callback - INFO - Completed processing callback for node 69d05ef8d2be3e1cf2a5ad9f 2026-04-04 00:58:01,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061ccd2be3e1cf2a5b18b HTTP/1.1" 200 None 2026-04-04 00:58:01,268 - lava_callback - INFO - Completed processing callback for node 69d061ccd2be3e1cf2a5b18b 2026-04-04 00:58:01,472 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:01,474 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c0d2be3e1cf2a5b154/lava_callback.json.gz 2026-04-04 00:58:01,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:01,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061c0d2be3e1cf2a5b154 HTTP/1.1" 200 None 2026-04-04 00:58:01,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:01,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:58:01,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:01,566 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:01,568 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d061ced2be3e1cf2a5b18e/lava-logs.yaml 2026-04-04 00:58:01,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:01,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061c0d2be3e1cf2a5b154 HTTP/1.1" 200 None 2026-04-04 00:58:01,600 - lava_callback - INFO - Completed processing callback for node 69d061c0d2be3e1cf2a5b154 2026-04-04 00:58:01,823 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:01,825 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-kontron-69d061ced2be3e1cf2a5b18e/lava_callback.json.gz 2026-04-04 00:58:01,870 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:01,884 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061ced2be3e1cf2a5b18e HTTP/1.1" 200 None 2026-04-04 00:58:01,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:01,900 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:58:01,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:01,941 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061ced2be3e1cf2a5b18e HTTP/1.1" 200 None 2026-04-04 00:58:01,946 - lava_callback - INFO - Completed processing callback for node 69d061ced2be3e1cf2a5b18e 2026-04-04 00:58:02,776 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:02,793 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:37004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37018 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:58:13,165 - lava_callback - INFO - Received callback for node 69d061d3d2be3e1cf2a5b19a INFO: 172.20.0.1:58308 - "POST /node/69d061d3d2be3e1cf2a5b19a HTTP/1.0" 202 Accepted 2026-04-04 00:58:13,172 - lava_callback - INFO - Starting async job submit for node 69d061d3d2be3e1cf2a5b19a 2026-04-04 00:58:13,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:13,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061d3d2be3e1cf2a5b19a HTTP/1.1" 200 None 2026-04-04 00:58:13,226 - lava_callback - DEBUG - Processing node 69d061d3d2be3e1cf2a5b19a: baseline-arm64-foundriesio 2026-04-04 00:58:13,668 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:13,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:13,805 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d061d3d2be3e1cf2a5b19a/log.txt.gz 2026-04-04 00:58:14,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:14,285 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:14,287 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d061d3d2be3e1cf2a5b19a/lava-logs.yaml 2026-04-04 00:58:14,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:14,468 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:14,470 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-foundriesio-69d061d3d2be3e1cf2a5b19a/lava_callback.json.gz 2026-04-04 00:58:14,521 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:14,535 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061d3d2be3e1cf2a5b19a HTTP/1.1" 200 None 2026-04-04 00:58:14,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:14,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:58:14,555 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:14,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061d3d2be3e1cf2a5b19a HTTP/1.1" 200 None 2026-04-04 00:58:14,595 - lava_callback - INFO - Completed processing callback for node 69d061d3d2be3e1cf2a5b19a INFO: 172.20.0.1:58316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58318 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:58:18,270 - lava_callback - INFO - Received callback for node 69d061bdd2be3e1cf2a5b14b INFO: 172.20.0.1:58330 - "POST /node/69d061bdd2be3e1cf2a5b14b HTTP/1.0" 202 Accepted 2026-04-04 00:58:18,276 - lava_callback - INFO - Starting async job submit for node 69d061bdd2be3e1cf2a5b14b 2026-04-04 00:58:18,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:18,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061bdd2be3e1cf2a5b14b HTTP/1.1" 200 None 2026-04-04 00:58:18,318 - lava_callback - DEBUG - Processing node 69d061bdd2be3e1cf2a5b14b: baseline-arm64-broonie 2026-04-04 00:58:18,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:18,743 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:18,745 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bdd2be3e1cf2a5b14b/log.txt.gz 2026-04-04 00:58:18,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:19,175 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:19,177 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bdd2be3e1cf2a5b14b/lava-logs.yaml 2026-04-04 00:58:19,184 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:19,320 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:19,322 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bdd2be3e1cf2a5b14b/lava_callback.json.gz 2026-04-04 00:58:19,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:19,358 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061bdd2be3e1cf2a5b14b HTTP/1.1" 200 None 2026-04-04 00:58:19,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:19,372 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:58:19,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:19,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061bdd2be3e1cf2a5b14b HTTP/1.1" 200 None 2026-04-04 00:58:19,406 - lava_callback - INFO - Completed processing callback for node 69d061bdd2be3e1cf2a5b14b 2026-04-04 00:58:19,724 - lava_callback - INFO - Received callback for node 69d061c3d2be3e1cf2a5b162 INFO: 172.20.0.1:32992 - "POST /node/69d061c3d2be3e1cf2a5b162 HTTP/1.0" 202 Accepted 2026-04-04 00:58:19,731 - lava_callback - INFO - Starting async job submit for node 69d061c3d2be3e1cf2a5b162 2026-04-04 00:58:19,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:19,800 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061c3d2be3e1cf2a5b162 HTTP/1.1" 200 None 2026-04-04 00:58:19,801 - lava_callback - DEBUG - Processing node 69d061c3d2be3e1cf2a5b162: baseline-arm64-broonie 2026-04-04 00:58:20,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:20,239 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:20,241 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c3d2be3e1cf2a5b162/log.txt.gz 2026-04-04 00:58:20,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:20,607 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:20,609 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c3d2be3e1cf2a5b162/lava-logs.yaml 2026-04-04 00:58:20,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:20,784 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:20,786 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c3d2be3e1cf2a5b162/lava_callback.json.gz 2026-04-04 00:58:20,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:20,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061c3d2be3e1cf2a5b162 HTTP/1.1" 200 None 2026-04-04 00:58:20,830 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:20,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:58:20,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:20,869 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061c3d2be3e1cf2a5b162 HTTP/1.1" 200 None 2026-04-04 00:58:20,871 - lava_callback - INFO - Completed processing callback for node 69d061c3d2be3e1cf2a5b162 INFO: 172.20.0.1:32996 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:58:26,898 - lava_callback - INFO - Received callback for node 69d061c9d2be3e1cf2a5b17e INFO: 172.20.0.1:33002 - "POST /node/69d061c9d2be3e1cf2a5b17e HTTP/1.0" 202 Accepted 2026-04-04 00:58:26,904 - lava_callback - INFO - Starting async job submit for node 69d061c9d2be3e1cf2a5b17e 2026-04-04 00:58:26,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:26,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061c9d2be3e1cf2a5b17e HTTP/1.1" 200 None 2026-04-04 00:58:26,949 - lava_callback - DEBUG - Processing node 69d061c9d2be3e1cf2a5b17e: baseline-arm64-clabbe 2026-04-04 00:58:27,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:27,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:27,303 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69d061c9d2be3e1cf2a5b17e/log.txt.gz 2026-04-04 00:58:27,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:27,650 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:27,652 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69d061c9d2be3e1cf2a5b17e/lava-logs.yaml 2026-04-04 00:58:27,658 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:27,811 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:27,813 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-clabbe-69d061c9d2be3e1cf2a5b17e/lava_callback.json.gz 2026-04-04 00:58:27,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:27,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061c9d2be3e1cf2a5b17e HTTP/1.1" 200 None 2026-04-04 00:58:27,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:27,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:58:27,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:27,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061c9d2be3e1cf2a5b17e HTTP/1.1" 200 None 2026-04-04 00:58:27,904 - lava_callback - INFO - Completed processing callback for node 69d061c9d2be3e1cf2a5b17e 2026-04-04 00:58:32,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:32,816 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-04 00:58:34,502 - lava_callback - INFO - Received callback for node 69cdbf3223eb053795d422cf INFO: 172.20.0.1:54524 - "POST /node/69cdbf3223eb053795d422cf HTTP/1.0" 202 Accepted 2026-04-04 00:58:34,507 - lava_callback - INFO - Starting async job submit for node 69cdbf3223eb053795d422cf 2026-04-04 00:58:34,508 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:34,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbf3223eb053795d422cf HTTP/1.1" 200 None 2026-04-04 00:58:34,524 - lava_callback - DEBUG - Processing node 69cdbf3223eb053795d422cf: baseline-arm-clabbe 2026-04-04 00:58:34,524 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbf3223eb053795d422cf 2026-04-04 00:58:34,525 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:34,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbf3223eb053795d422cf HTTP/1.1" 200 None 2026-04-04 00:58:34,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:34,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-04 00:58:34,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:34,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbf3223eb053795d422cf HTTP/1.1" 200 None 2026-04-04 00:58:34,608 - lava_callback - ERROR - Error processing callback for node 69cdbf3223eb053795d422cf: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54536 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:58:38,278 - lava_callback - INFO - Received callback for node 69d061c2d2be3e1cf2a5b15d INFO: 172.20.0.1:54548 - "POST /node/69d061c2d2be3e1cf2a5b15d HTTP/1.0" 202 Accepted 2026-04-04 00:58:38,284 - lava_callback - INFO - Starting async job submit for node 69d061c2d2be3e1cf2a5b15d 2026-04-04 00:58:38,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:38,330 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061c2d2be3e1cf2a5b15d HTTP/1.1" 200 None 2026-04-04 00:58:38,331 - lava_callback - DEBUG - Processing node 69d061c2d2be3e1cf2a5b15d: baseline-arm64-broonie 2026-04-04 00:58:38,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:38,889 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:38,891 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c2d2be3e1cf2a5b15d/log.txt.gz 2026-04-04 00:58:39,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:39,287 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:39,289 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c2d2be3e1cf2a5b15d/lava-logs.yaml 2026-04-04 00:58:39,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:39,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:39,437 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c2d2be3e1cf2a5b15d/lava_callback.json.gz 2026-04-04 00:58:39,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:39,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061c2d2be3e1cf2a5b15d HTTP/1.1" 200 None 2026-04-04 00:58:39,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:39,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:58:39,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:39,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061c2d2be3e1cf2a5b15d HTTP/1.1" 200 None 2026-04-04 00:58:39,520 - lava_callback - INFO - Completed processing callback for node 69d061c2d2be3e1cf2a5b15d 2026-04-04 00:58:39,755 - lava_callback - INFO - Received callback for node 69d061c5d2be3e1cf2a5b16b INFO: 172.20.0.1:54586 - "POST /node/69d061c5d2be3e1cf2a5b16b HTTP/1.0" 202 Accepted 2026-04-04 00:58:39,762 - lava_callback - INFO - Starting async job submit for node 69d061c5d2be3e1cf2a5b16b 2026-04-04 00:58:39,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:39,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061c5d2be3e1cf2a5b16b HTTP/1.1" 200 None 2026-04-04 00:58:39,800 - lava_callback - DEBUG - Processing node 69d061c5d2be3e1cf2a5b16b: baseline-arm64-broonie 2026-04-04 00:58:40,065 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:40,185 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:40,186 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c5d2be3e1cf2a5b16b/log.txt.gz 2026-04-04 00:58:40,346 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:40,535 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:40,537 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c5d2be3e1cf2a5b16b/lava-logs.yaml 2026-04-04 00:58:40,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:40,677 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:40,679 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061c5d2be3e1cf2a5b16b/lava_callback.json.gz 2026-04-04 00:58:40,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:40,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061c5d2be3e1cf2a5b16b HTTP/1.1" 200 None 2026-04-04 00:58:40,741 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:40,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:58:40,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:40,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061c5d2be3e1cf2a5b16b HTTP/1.1" 200 None 2026-04-04 00:58:40,800 - lava_callback - INFO - Completed processing callback for node 69d061c5d2be3e1cf2a5b16b 2026-04-04 00:58:46,388 - lava_callback - INFO - Received callback for node 69d061cfd2be3e1cf2a5b191 INFO: 172.20.0.1:54590 - "POST /node/69d061cfd2be3e1cf2a5b191 HTTP/1.0" 202 Accepted 2026-04-04 00:58:46,394 - lava_callback - INFO - Starting async job submit for node 69d061cfd2be3e1cf2a5b191 2026-04-04 00:58:46,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:46,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061cfd2be3e1cf2a5b191 HTTP/1.1" 200 None 2026-04-04 00:58:46,430 - lava_callback - DEBUG - Processing node 69d061cfd2be3e1cf2a5b191: baseline-arm64-pengutronix INFO: 172.20.0.1:54598 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:58:46,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:46,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:46,726 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d061cfd2be3e1cf2a5b191/log.txt.gz 2026-04-04 00:58:46,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:47,064 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:47,066 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d061cfd2be3e1cf2a5b191/lava-logs.yaml 2026-04-04 00:58:47,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:47,243 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:47,245 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d061cfd2be3e1cf2a5b191/lava_callback.json.gz 2026-04-04 00:58:47,312 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:47,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061cfd2be3e1cf2a5b191 HTTP/1.1" 200 None 2026-04-04 00:58:47,327 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:47,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:58:47,343 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:47,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061cfd2be3e1cf2a5b191 HTTP/1.1" 200 None 2026-04-04 00:58:47,414 - lava_callback - INFO - Completed processing callback for node 69d061cfd2be3e1cf2a5b191 2026-04-04 00:58:50,923 - lava_callback - INFO - Received callback for node 69d0622fd2be3e1cf2a5b2f9 INFO: 172.20.0.1:35838 - "POST /node/69d0622fd2be3e1cf2a5b2f9 HTTP/1.0" 202 Accepted 2026-04-04 00:58:50,929 - lava_callback - INFO - Starting async job submit for node 69d0622fd2be3e1cf2a5b2f9 2026-04-04 00:58:50,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:50,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0622fd2be3e1cf2a5b2f9 HTTP/1.1" 200 None 2026-04-04 00:58:50,948 - lava_callback - DEBUG - Processing node 69d0622fd2be3e1cf2a5b2f9: baseline-arm 2026-04-04 00:58:50,953 - lava_callback - WARNING - Failed to upload log 2026-04-04 00:58:50,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:51,096 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:51,098 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d0622fd2be3e1cf2a5b2f9/lava-logs.yaml 2026-04-04 00:58:51,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:51,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:51,239 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d0622fd2be3e1cf2a5b2f9/lava_callback.json.gz 2026-04-04 00:58:51,241 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:51,256 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0622fd2be3e1cf2a5b2f9 HTTP/1.1" 200 None 2026-04-04 00:58:51,257 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:51,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 00:58:51,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:51,296 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0622fd2be3e1cf2a5b2f9 HTTP/1.1" 200 None 2026-04-04 00:58:51,308 - lava_callback - INFO - Completed processing callback for node 69d0622fd2be3e1cf2a5b2f9 INFO: 172.20.0.1:35848 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:58:57,513 - lava_callback - INFO - Received callback for node 69d061bad2be3e1cf2a5b117 INFO: 172.20.0.1:35850 - "POST /node/69d061bad2be3e1cf2a5b117 HTTP/1.0" 202 Accepted 2026-04-04 00:58:57,520 - lava_callback - INFO - Starting async job submit for node 69d061bad2be3e1cf2a5b117 2026-04-04 00:58:57,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:57,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061bad2be3e1cf2a5b117 HTTP/1.1" 200 None 2026-04-04 00:58:57,563 - lava_callback - DEBUG - Processing node 69d061bad2be3e1cf2a5b117: baseline-arm64-broonie 2026-04-04 00:58:57,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:57,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:57,994 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bad2be3e1cf2a5b117/log.txt.gz 2026-04-04 00:58:58,134 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:58,317 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:58,319 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bad2be3e1cf2a5b117/lava-logs.yaml 2026-04-04 00:58:58,325 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:58:58,456 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:58:58,458 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bad2be3e1cf2a5b117/lava_callback.json.gz 2026-04-04 00:58:58,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:58,494 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061bad2be3e1cf2a5b117 HTTP/1.1" 200 None 2026-04-04 00:58:58,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:58,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:58:58,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:58:58,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061bad2be3e1cf2a5b117 HTTP/1.1" 200 None 2026-04-04 00:58:58,536 - lava_callback - INFO - Completed processing callback for node 69d061bad2be3e1cf2a5b117 2026-04-04 00:59:02,818 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:59:02,836 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:48832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37690 - "GET /node/694afe236c96ed51d3172c6d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37706 - "GET /node/698ee510f0e3ffbba67d20f8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37714 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:59:54,534 - lava_callback - INFO - Received callback for node 69cdbf7d23eb053795d422f7 INFO: 172.20.0.1:48994 - "POST /node/69cdbf7d23eb053795d422f7 HTTP/1.0" 202 Accepted 2026-04-04 00:59:54,540 - lava_callback - INFO - Starting async job submit for node 69cdbf7d23eb053795d422f7 2026-04-04 00:59:54,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:59:54,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbf7d23eb053795d422f7 HTTP/1.1" 200 None 2026-04-04 00:59:54,559 - lava_callback - DEBUG - Processing node 69cdbf7d23eb053795d422f7: baseline-arm-clabbe 2026-04-04 00:59:54,560 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdbf7d23eb053795d422f7 2026-04-04 00:59:54,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:59:54,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdbf7d23eb053795d422f7 HTTP/1.1" 200 None 2026-04-04 00:59:54,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:59:54,593 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc292cb62322ad965da7 HTTP/1.1" 200 None 2026-04-04 00:59:54,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:59:54,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdbf7d23eb053795d422f7 HTTP/1.1" 200 None 2026-04-04 00:59:54,620 - lava_callback - ERROR - Error processing callback for node 69cdbf7d23eb053795d422f7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48996 - "GET / HTTP/1.0" 200 OK 2026-04-04 00:59:57,530 - lava_callback - INFO - Received callback for node 69d061bfd2be3e1cf2a5b150 INFO: 172.20.0.1:49000 - "POST /node/69d061bfd2be3e1cf2a5b150 HTTP/1.0" 202 Accepted 2026-04-04 00:59:57,536 - lava_callback - INFO - Starting async job submit for node 69d061bfd2be3e1cf2a5b150 2026-04-04 00:59:57,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:59:57,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061bfd2be3e1cf2a5b150 HTTP/1.1" 200 None 2026-04-04 00:59:57,581 - lava_callback - DEBUG - Processing node 69d061bfd2be3e1cf2a5b150: baseline-arm64-broonie 2026-04-04 00:59:57,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:59:58,015 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:59:58,016 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bfd2be3e1cf2a5b150/log.txt.gz 2026-04-04 00:59:58,221 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:59:58,439 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:59:58,441 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bfd2be3e1cf2a5b150/lava-logs.yaml 2026-04-04 00:59:58,449 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 00:59:58,631 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 00:59:58,633 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bfd2be3e1cf2a5b150/lava_callback.json.gz 2026-04-04 00:59:58,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:59:58,684 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061bfd2be3e1cf2a5b150 HTTP/1.1" 200 None 2026-04-04 00:59:58,685 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:59:58,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 00:59:58,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 00:59:58,741 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061bfd2be3e1cf2a5b150 HTTP/1.1" 200 None 2026-04-04 00:59:58,748 - lava_callback - INFO - Completed processing callback for node 69d061bfd2be3e1cf2a5b150 2026-04-04 01:00:02,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:00:02,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34930 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39700 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:01:39,821 - lava_callback - INFO - Received callback for node 69d05eafd2be3e1cf2a5ac4c INFO: 172.20.0.1:45766 - "POST /node/69d05eafd2be3e1cf2a5ac4c HTTP/1.0" 202 Accepted 2026-04-04 01:01:39,827 - lava_callback - INFO - Starting async job submit for node 69d05eafd2be3e1cf2a5ac4c 2026-04-04 01:01:39,881 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:01:39,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eafd2be3e1cf2a5ac4c HTTP/1.1" 200 None 2026-04-04 01:01:39,901 - lava_callback - DEBUG - Processing node 69d05eafd2be3e1cf2a5ac4c: baseline-arm 2026-04-04 01:01:40,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:01:40,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:01:40,687 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eafd2be3e1cf2a5ac4c/log.txt.gz 2026-04-04 01:01:41,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:01:41,394 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:01:41,397 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eafd2be3e1cf2a5ac4c/lava-logs.yaml 2026-04-04 01:01:41,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:01:41,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:01:41,562 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d05eafd2be3e1cf2a5ac4c/lava_callback.json.gz DEBUG: Setup failed for 69d05eafd2be3e1cf2a5ac4c. Transitting job node result: incomplete -> fail 2026-04-04 01:01:41,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:01:41,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05eafd2be3e1cf2a5ac4c HTTP/1.1" 200 None 2026-04-04 01:01:41,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:01:41,714 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 01:01:41,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:01:41,739 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05eafd2be3e1cf2a5ac4c HTTP/1.1" 200 None 2026-04-04 01:01:41,842 - lava_callback - INFO - Completed processing callback for node 69d05eafd2be3e1cf2a5ac4c INFO: 172.20.0.1:45776 - "GET /node/69bd977a0e93e231472417c0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43134 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:02:02,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:02:02,877 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41912 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:03:02,173 - lava_callback - INFO - Received callback for node 69d06193d2be3e1cf2a5b086 INFO: 172.20.0.1:55052 - "POST /node/69d06193d2be3e1cf2a5b086 HTTP/1.0" 202 Accepted 2026-04-04 01:03:02,179 - lava_callback - INFO - Starting async job submit for node 69d06193d2be3e1cf2a5b086 2026-04-04 01:03:02,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:02,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06193d2be3e1cf2a5b086 HTTP/1.1" 200 None 2026-04-04 01:03:02,218 - lava_callback - DEBUG - Processing node 69d06193d2be3e1cf2a5b086: baseline-arm64 2026-04-04 01:03:02,575 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:02,608 - lava_callback - INFO - Received callback for node 69d0619cd2be3e1cf2a5b09d INFO: 172.20.0.1:55066 - "POST /node/69d0619cd2be3e1cf2a5b09d HTTP/1.0" 202 Accepted 2026-04-04 01:03:02,618 - lava_callback - INFO - Starting async job submit for node 69d0619cd2be3e1cf2a5b09d 2026-04-04 01:03:02,653 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:02,668 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0619cd2be3e1cf2a5b09d HTTP/1.1" 200 None 2026-04-04 01:03:02,670 - lava_callback - DEBUG - Processing node 69d0619cd2be3e1cf2a5b09d: baseline-arm64 2026-04-04 01:03:02,750 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:02,811 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06193d2be3e1cf2a5b086/log.txt.gz 2026-04-04 01:03:03,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:03,178 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:03,307 - lava_callback - INFO - Received callback for node 69d0618ed2be3e1cf2a5b07a INFO: 172.20.0.1:55072 - "POST /node/69d0618ed2be3e1cf2a5b07a HTTP/1.0" 202 Accepted 2026-04-04 01:03:03,319 - lava_callback - INFO - Starting async job submit for node 69d0618ed2be3e1cf2a5b07a 2026-04-04 01:03:03,336 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:03,359 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:03,360 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619cd2be3e1cf2a5b09d/log.txt.gz 2026-04-04 01:03:03,425 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618ed2be3e1cf2a5b07a HTTP/1.1" 200 None 2026-04-04 01:03:03,436 - lava_callback - DEBUG - Processing node 69d0618ed2be3e1cf2a5b07a: baseline-arm64 2026-04-04 01:03:03,441 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:03,560 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06193d2be3e1cf2a5b086/lava-logs.yaml 2026-04-04 01:03:03,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:03,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:03,760 - lava_callback - WARNING - Failed to upload log 2026-04-04 01:03:03,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:03,907 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:03,909 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06193d2be3e1cf2a5b086/lava_callback.json.gz 2026-04-04 01:03:03,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:03,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06193d2be3e1cf2a5b086 HTTP/1.1" 200 None 2026-04-04 01:03:03,960 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:03,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:03,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:03,978 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:03,980 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619cd2be3e1cf2a5b09d/lava-logs.yaml 2026-04-04 01:03:03,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:04,001 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:04,003 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618ed2be3e1cf2a5b07a/lava-logs.yaml 2026-04-04 01:03:04,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:04,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06193d2be3e1cf2a5b086 HTTP/1.1" 200 None 2026-04-04 01:03:04,025 - lava_callback - INFO - Completed processing callback for node 69d06193d2be3e1cf2a5b086 2026-04-04 01:03:04,206 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:04,209 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619cd2be3e1cf2a5b09d/lava_callback.json.gz 2026-04-04 01:03:04,229 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:04,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:04,242 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618ed2be3e1cf2a5b07a/lava_callback.json.gz 2026-04-04 01:03:04,276 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:04,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0619cd2be3e1cf2a5b09d HTTP/1.1" 200 None 2026-04-04 01:03:04,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:04,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618ed2be3e1cf2a5b07a HTTP/1.1" 200 None 2026-04-04 01:03:04,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:04,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:04,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:04,316 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:04,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:04,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0619cd2be3e1cf2a5b09d HTTP/1.1" 200 None 2026-04-04 01:03:04,349 - lava_callback - INFO - Completed processing callback for node 69d0619cd2be3e1cf2a5b09d 2026-04-04 01:03:04,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0618ed2be3e1cf2a5b07a HTTP/1.1" 200 None 2026-04-04 01:03:04,433 - lava_callback - INFO - Completed processing callback for node 69d0618ed2be3e1cf2a5b07a INFO: 172.20.0.1:55078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55082 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:03:07,992 - lava_callback - INFO - Received callback for node 69d06198d2be3e1cf2a5b092 INFO: 172.20.0.1:55094 - "POST /node/69d06198d2be3e1cf2a5b092 HTTP/1.0" 202 Accepted 2026-04-04 01:03:07,999 - lava_callback - INFO - Starting async job submit for node 69d06198d2be3e1cf2a5b092 2026-04-04 01:03:08,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:08,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06198d2be3e1cf2a5b092 HTTP/1.1" 200 None 2026-04-04 01:03:08,040 - lava_callback - DEBUG - Processing node 69d06198d2be3e1cf2a5b092: baseline-arm64 2026-04-04 01:03:08,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:08,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:08,410 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06198d2be3e1cf2a5b092/log.txt.gz 2026-04-04 01:03:08,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:08,724 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:08,726 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06198d2be3e1cf2a5b092/lava-logs.yaml 2026-04-04 01:03:08,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:08,921 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:08,923 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06198d2be3e1cf2a5b092/lava_callback.json.gz 2026-04-04 01:03:08,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:08,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06198d2be3e1cf2a5b092 HTTP/1.1" 200 None 2026-04-04 01:03:08,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:08,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:08,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:09,003 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06198d2be3e1cf2a5b092 HTTP/1.1" 200 None 2026-04-04 01:03:09,004 - lava_callback - INFO - Completed processing callback for node 69d06198d2be3e1cf2a5b092 2026-04-04 01:03:10,518 - lava_callback - INFO - Received callback for node 69d06192d2be3e1cf2a5b083 INFO: 172.20.0.1:44346 - "POST /node/69d06192d2be3e1cf2a5b083 HTTP/1.0" 202 Accepted 2026-04-04 01:03:10,524 - lava_callback - INFO - Starting async job submit for node 69d06192d2be3e1cf2a5b083 2026-04-04 01:03:10,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:10,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06192d2be3e1cf2a5b083 HTTP/1.1" 200 None 2026-04-04 01:03:10,573 - lava_callback - DEBUG - Processing node 69d06192d2be3e1cf2a5b083: baseline-arm64 2026-04-04 01:03:10,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:11,029 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:11,031 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06192d2be3e1cf2a5b083/log.txt.gz 2026-04-04 01:03:11,189 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:11,365 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:11,367 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06192d2be3e1cf2a5b083/lava-logs.yaml 2026-04-04 01:03:11,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:11,537 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:11,538 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06192d2be3e1cf2a5b083/lava_callback.json.gz 2026-04-04 01:03:11,561 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:11,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06192d2be3e1cf2a5b083 HTTP/1.1" 200 None 2026-04-04 01:03:11,577 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:11,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:11,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:11,617 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06192d2be3e1cf2a5b083 HTTP/1.1" 200 None 2026-04-04 01:03:11,624 - lava_callback - INFO - Completed processing callback for node 69d06192d2be3e1cf2a5b083 2026-04-04 01:03:13,272 - lava_callback - INFO - Received callback for node 69cdc04d23eb053795d42337 INFO: 172.20.0.1:44348 - "POST /node/69cdc04d23eb053795d42337 HTTP/1.0" 202 Accepted 2026-04-04 01:03:13,277 - lava_callback - INFO - Starting async job submit for node 69cdc04d23eb053795d42337 2026-04-04 01:03:13,278 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:13,291 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc04d23eb053795d42337 HTTP/1.1" 200 None 2026-04-04 01:03:13,292 - lava_callback - DEBUG - Processing node 69cdc04d23eb053795d42337: baseline-arm-baylibre 2026-04-04 01:03:13,292 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc04d23eb053795d42337 2026-04-04 01:03:13,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:13,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc04d23eb053795d42337 HTTP/1.1" 200 None 2026-04-04 01:03:13,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:13,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 01:03:13,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:13,347 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc04d23eb053795d42337 HTTP/1.1" 200 None 2026-04-04 01:03:13,349 - lava_callback - ERROR - Error processing callback for node 69cdc04d23eb053795d42337: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44364 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:03:18,758 - lava_callback - INFO - Received callback for node 69d061bcd2be3e1cf2a5b11a INFO: 172.20.0.1:44376 - "POST /node/69d061bcd2be3e1cf2a5b11a HTTP/1.0" 202 Accepted 2026-04-04 01:03:18,769 - lava_callback - INFO - Starting async job submit for node 69d061bcd2be3e1cf2a5b11a 2026-04-04 01:03:18,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:18,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061bcd2be3e1cf2a5b11a HTTP/1.1" 200 None 2026-04-04 01:03:18,861 - lava_callback - DEBUG - Processing node 69d061bcd2be3e1cf2a5b11a: baseline-arm64-broonie 2026-04-04 01:03:19,769 - lava_callback - INFO - Received callback for node 69d0618fd2be3e1cf2a5b07d INFO: 172.20.0.1:37590 - "POST /node/69d0618fd2be3e1cf2a5b07d HTTP/1.0" 202 Accepted 2026-04-04 01:03:19,797 - lava_callback - INFO - Starting async job submit for node 69d0618fd2be3e1cf2a5b07d 2026-04-04 01:03:19,851 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:19,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:19,886 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618fd2be3e1cf2a5b07d HTTP/1.1" 200 None 2026-04-04 01:03:19,888 - lava_callback - DEBUG - Processing node 69d0618fd2be3e1cf2a5b07d: baseline-arm64 2026-04-04 01:03:20,335 - lava_callback - WARNING - Failed to upload log 2026-04-04 01:03:20,359 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:20,366 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bcd2be3e1cf2a5b11a/log.txt.gz 2026-04-04 01:03:20,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:20,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:20,956 - lava_callback - INFO - Received callback for node 69d0619ad2be3e1cf2a5b098 INFO: 172.20.0.1:37592 - "POST /node/69d0619ad2be3e1cf2a5b098 HTTP/1.0" 202 Accepted 2026-04-04 01:03:20,967 - lava_callback - INFO - Starting async job submit for node 69d0619ad2be3e1cf2a5b098 2026-04-04 01:03:21,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:21,033 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0619ad2be3e1cf2a5b098 HTTP/1.1" 200 None 2026-04-04 01:03:21,034 - lava_callback - DEBUG - Processing node 69d0619ad2be3e1cf2a5b098: baseline-arm64 2026-04-04 01:03:21,160 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:21,192 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:21,199 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618fd2be3e1cf2a5b07d/lava-logs.yaml 2026-04-04 01:03:21,221 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bcd2be3e1cf2a5b11a/lava-logs.yaml 2026-04-04 01:03:21,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:21,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:21,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:21,754 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:21,756 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618fd2be3e1cf2a5b07d/lava_callback.json.gz 2026-04-04 01:03:21,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:21,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618fd2be3e1cf2a5b07d HTTP/1.1" 200 None 2026-04-04 01:03:21,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:21,879 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:21,882 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d061bcd2be3e1cf2a5b11a/lava_callback.json.gz 2026-04-04 01:03:21,893 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:21,895 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619ad2be3e1cf2a5b098/log.txt.gz 2026-04-04 01:03:21,935 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:21,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 DEBUG: 69d061bcd2be3e1cf2a5b11a Transitting job node result: pass -> incomplete 2026-04-04 01:03:22,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:22,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0618fd2be3e1cf2a5b07d HTTP/1.1" 200 None 2026-04-04 01:03:22,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061bcd2be3e1cf2a5b11a HTTP/1.1" 200 None 2026-04-04 01:03:22,217 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:22,308 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:22,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:22,319 - lava_callback - INFO - Completed processing callback for node 69d0618fd2be3e1cf2a5b07d 2026-04-04 01:03:22,410 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061bcd2be3e1cf2a5b11a HTTP/1.1" 200 None 2026-04-04 01:03:22,412 - lava_callback - INFO - Completed processing callback for node 69d061bcd2be3e1cf2a5b11a 2026-04-04 01:03:22,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:22,924 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:22,927 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619ad2be3e1cf2a5b098/lava-logs.yaml 2026-04-04 01:03:22,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:23,138 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:23,140 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619ad2be3e1cf2a5b098/lava_callback.json.gz 2026-04-04 01:03:23,175 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:23,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0619ad2be3e1cf2a5b098 HTTP/1.1" 200 None 2026-04-04 01:03:23,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:23,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:23,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:23,275 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0619ad2be3e1cf2a5b098 HTTP/1.1" 200 None 2026-04-04 01:03:23,277 - lava_callback - INFO - Completed processing callback for node 69d0619ad2be3e1cf2a5b098 2026-04-04 01:03:25,624 - lava_callback - INFO - Received callback for node 69d061acd2be3e1cf2a5b0f1 INFO: 172.20.0.1:37606 - "POST /node/69d061acd2be3e1cf2a5b0f1 HTTP/1.0" 202 Accepted 2026-04-04 01:03:25,631 - lava_callback - INFO - Starting async job submit for node 69d061acd2be3e1cf2a5b0f1 2026-04-04 01:03:25,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:25,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061acd2be3e1cf2a5b0f1 HTTP/1.1" 200 None 2026-04-04 01:03:25,686 - lava_callback - DEBUG - Processing node 69d061acd2be3e1cf2a5b0f1: baseline-nfs-arm64 2026-04-04 01:03:26,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:26,331 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:26,333 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061acd2be3e1cf2a5b0f1/log.txt.gz INFO: 172.20.0.1:37622 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:03:26,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:27,115 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:27,117 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061acd2be3e1cf2a5b0f1/lava-logs.yaml 2026-04-04 01:03:27,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:27,604 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:27,606 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061acd2be3e1cf2a5b0f1/lava_callback.json.gz 2026-04-04 01:03:27,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:27,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061acd2be3e1cf2a5b0f1 HTTP/1.1" 200 None 2026-04-04 01:03:27,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:27,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:27,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:27,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061acd2be3e1cf2a5b0f1 HTTP/1.1" 200 None 2026-04-04 01:03:27,742 - lava_callback - INFO - Completed processing callback for node 69d061acd2be3e1cf2a5b0f1 2026-04-04 01:03:28,518 - lava_callback - INFO - Received callback for node 69d0618dd2be3e1cf2a5b077 INFO: 172.20.0.1:37638 - "POST /node/69d0618dd2be3e1cf2a5b077 HTTP/1.0" 202 Accepted 2026-04-04 01:03:28,530 - lava_callback - INFO - Starting async job submit for node 69d0618dd2be3e1cf2a5b077 2026-04-04 01:03:28,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:28,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618dd2be3e1cf2a5b077 HTTP/1.1" 200 None 2026-04-04 01:03:28,591 - lava_callback - DEBUG - Processing node 69d0618dd2be3e1cf2a5b077: baseline-arm64 2026-04-04 01:03:29,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:29,162 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:29,164 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618dd2be3e1cf2a5b077/log.txt.gz 2026-04-04 01:03:29,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:29,870 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:29,872 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618dd2be3e1cf2a5b077/lava-logs.yaml 2026-04-04 01:03:29,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:29,950 - lava_callback - INFO - Received callback for node 69d061a5d2be3e1cf2a5b0df INFO: 172.20.0.1:45372 - "POST /node/69d061a5d2be3e1cf2a5b0df HTTP/1.0" 202 Accepted 2026-04-04 01:03:29,959 - lava_callback - INFO - Starting async job submit for node 69d061a5d2be3e1cf2a5b0df 2026-04-04 01:03:29,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:30,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a5d2be3e1cf2a5b0df HTTP/1.1" 200 None 2026-04-04 01:03:30,033 - lava_callback - DEBUG - Processing node 69d061a5d2be3e1cf2a5b0df: baseline-nfs-arm64 2026-04-04 01:03:30,390 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:30,393 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0618dd2be3e1cf2a5b077/lava_callback.json.gz 2026-04-04 01:03:30,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:30,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0618dd2be3e1cf2a5b077 HTTP/1.1" 200 None 2026-04-04 01:03:30,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:30,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:30,732 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:30,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0618dd2be3e1cf2a5b077 HTTP/1.1" 200 None 2026-04-04 01:03:30,925 - lava_callback - INFO - Completed processing callback for node 69d0618dd2be3e1cf2a5b077 2026-04-04 01:03:30,923 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:31,251 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:31,254 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a5d2be3e1cf2a5b0df/log.txt.gz 2026-04-04 01:03:31,558 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:32,013 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:32,015 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a5d2be3e1cf2a5b0df/lava-logs.yaml 2026-04-04 01:03:32,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:32,163 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:32,165 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a5d2be3e1cf2a5b0df/lava_callback.json.gz 2026-04-04 01:03:32,195 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:32,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a5d2be3e1cf2a5b0df HTTP/1.1" 200 None 2026-04-04 01:03:32,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:32,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:32,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:32,292 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061a5d2be3e1cf2a5b0df HTTP/1.1" 200 None 2026-04-04 01:03:32,294 - lava_callback - INFO - Completed processing callback for node 69d061a5d2be3e1cf2a5b0df 2026-04-04 01:03:32,531 - lava_callback - INFO - Received callback for node 69d061a6d2be3e1cf2a5b0e2 INFO: 172.20.0.1:45378 - "POST /node/69d061a6d2be3e1cf2a5b0e2 HTTP/1.0" 202 Accepted 2026-04-04 01:03:32,544 - lava_callback - INFO - Starting async job submit for node 69d061a6d2be3e1cf2a5b0e2 2026-04-04 01:03:32,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:32,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a6d2be3e1cf2a5b0e2 HTTP/1.1" 200 None 2026-04-04 01:03:32,633 - lava_callback - DEBUG - Processing node 69d061a6d2be3e1cf2a5b0e2: baseline-nfs-arm64 2026-04-04 01:03:32,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:33,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 2026-04-04 01:03:33,179 - lava_callback - INFO - Received callback for node 69d06194d2be3e1cf2a5b089 INFO: 172.20.0.1:45382 - "POST /node/69d06194d2be3e1cf2a5b089 HTTP/1.0" 202 Accepted 2026-04-04 01:03:33,217 - lava_callback - INFO - Starting async job submit for node 69d06194d2be3e1cf2a5b089 2026-04-04 01:03:33,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:33,385 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06194d2be3e1cf2a5b089 HTTP/1.1" 200 None 2026-04-04 01:03:33,386 - lava_callback - DEBUG - Processing node 69d06194d2be3e1cf2a5b089: baseline-arm64 2026-04-04 01:03:33,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:34,163 - lava_callback - INFO - Received callback for node 69d061aad2be3e1cf2a5b0ee INFO: 172.20.0.1:45394 - "POST /node/69d061aad2be3e1cf2a5b0ee HTTP/1.0" 202 Accepted 2026-04-04 01:03:34,179 - lava_callback - INFO - Starting async job submit for node 69d061aad2be3e1cf2a5b0ee 2026-04-04 01:03:34,266 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:34,293 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:34,327 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a6d2be3e1cf2a5b0e2/log.txt.gz 2026-04-04 01:03:34,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061aad2be3e1cf2a5b0ee HTTP/1.1" 200 None 2026-04-04 01:03:34,415 - lava_callback - DEBUG - Processing node 69d061aad2be3e1cf2a5b0ee: baseline-nfs-arm64 2026-04-04 01:03:34,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:34,993 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:35,070 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06194d2be3e1cf2a5b089/log.txt.gz 2026-04-04 01:03:35,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:35,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:35,980 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:35,985 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:35,985 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061aad2be3e1cf2a5b0ee/log.txt.gz 2026-04-04 01:03:36,008 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:36,010 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a6d2be3e1cf2a5b0e2/lava-logs.yaml 2026-04-04 01:03:36,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:36,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:45408 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:03:36,622 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:36,624 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a6d2be3e1cf2a5b0e2/lava_callback.json.gz 2026-04-04 01:03:36,658 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:36,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:36,660 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06194d2be3e1cf2a5b089/lava-logs.yaml 2026-04-04 01:03:36,671 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:36,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a6d2be3e1cf2a5b0e2 HTTP/1.1" 200 None 2026-04-04 01:03:36,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:36,683 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:36,685 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061aad2be3e1cf2a5b0ee/lava-logs.yaml 2026-04-04 01:03:36,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:36,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:36,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:36,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061a6d2be3e1cf2a5b0e2 HTTP/1.1" 200 None 2026-04-04 01:03:36,755 - lava_callback - INFO - Completed processing callback for node 69d061a6d2be3e1cf2a5b0e2 2026-04-04 01:03:37,190 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:37,192 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06194d2be3e1cf2a5b089/lava_callback.json.gz 2026-04-04 01:03:37,244 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:37,268 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06194d2be3e1cf2a5b089 HTTP/1.1" 200 None 2026-04-04 01:03:37,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:37,296 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:37,298 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061aad2be3e1cf2a5b0ee/lava_callback.json.gz 2026-04-04 01:03:37,317 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:37,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:37,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:37,351 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061aad2be3e1cf2a5b0ee HTTP/1.1" 200 None 2026-04-04 01:03:37,353 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:37,367 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06194d2be3e1cf2a5b089 HTTP/1.1" 200 None 2026-04-04 01:03:37,369 - lava_callback - INFO - Completed processing callback for node 69d06194d2be3e1cf2a5b089 2026-04-04 01:03:37,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:37,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:37,465 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061aad2be3e1cf2a5b0ee HTTP/1.1" 200 None 2026-04-04 01:03:37,467 - lava_callback - INFO - Completed processing callback for node 69d061aad2be3e1cf2a5b0ee 2026-04-04 01:03:39,610 - lava_callback - INFO - Received callback for node 69d0619dd2be3e1cf2a5b0a0 INFO: 172.20.0.1:45410 - "POST /node/69d0619dd2be3e1cf2a5b0a0 HTTP/1.0" 202 Accepted 2026-04-04 01:03:39,624 - lava_callback - INFO - Starting async job submit for node 69d0619dd2be3e1cf2a5b0a0 2026-04-04 01:03:39,661 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:39,689 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0619dd2be3e1cf2a5b0a0 HTTP/1.1" 200 None 2026-04-04 01:03:39,691 - lava_callback - DEBUG - Processing node 69d0619dd2be3e1cf2a5b0a0: baseline-arm64 2026-04-04 01:03:39,943 - lava_callback - INFO - Received callback for node 69d061a2d2be3e1cf2a5b0d9 INFO: 172.20.0.1:58490 - "POST /node/69d061a2d2be3e1cf2a5b0d9 HTTP/1.0" 202 Accepted 2026-04-04 01:03:39,974 - lava_callback - INFO - Starting async job submit for node 69d061a2d2be3e1cf2a5b0d9 2026-04-04 01:03:40,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:40,121 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a2d2be3e1cf2a5b0d9 HTTP/1.1" 200 None 2026-04-04 01:03:40,122 - lava_callback - DEBUG - Processing node 69d061a2d2be3e1cf2a5b0d9: baseline-nfs-arm64 2026-04-04 01:03:40,368 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:40,584 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:40,637 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619dd2be3e1cf2a5b0a0/log.txt.gz 2026-04-04 01:03:41,051 - lava_callback - INFO - Received callback for node 69d05ed5d2be3e1cf2a5acf4 INFO: 172.20.0.1:58492 - "POST /node/69d05ed5d2be3e1cf2a5acf4 HTTP/1.0" 202 Accepted 2026-04-04 01:03:41,108 - lava_callback - INFO - Starting async job submit for node 69d05ed5d2be3e1cf2a5acf4 2026-04-04 01:03:41,168 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:41,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:41,181 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:41,199 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ed5d2be3e1cf2a5acf4 HTTP/1.1" 200 None 2026-04-04 01:03:41,201 - lava_callback - DEBUG - Processing node 69d05ed5d2be3e1cf2a5acf4: baseline-arm-pengutronix 2026-04-04 01:03:41,373 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:41,690 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:41,693 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a2d2be3e1cf2a5b0d9/log.txt.gz 2026-04-04 01:03:41,726 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:41,750 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed5d2be3e1cf2a5acf4/log.txt.gz 2026-04-04 01:03:41,803 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:41,885 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619dd2be3e1cf2a5b0a0/lava-logs.yaml 2026-04-04 01:03:41,886 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:41,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:42,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:42,476 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:42,478 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619dd2be3e1cf2a5b0a0/lava_callback.json.gz 2026-04-04 01:03:42,514 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:42,516 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed5d2be3e1cf2a5acf4/lava-logs.yaml 2026-04-04 01:03:42,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:42,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:42,575 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:42,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0619dd2be3e1cf2a5b0a0 HTTP/1.1" 200 None 2026-04-04 01:03:42,585 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a2d2be3e1cf2a5b0d9/lava-logs.yaml 2026-04-04 01:03:42,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:42,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:42,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:42,622 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:42,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0619dd2be3e1cf2a5b0a0 HTTP/1.1" 200 None 2026-04-04 01:03:42,737 - lava_callback - INFO - Completed processing callback for node 69d0619dd2be3e1cf2a5b0a0 2026-04-04 01:03:43,186 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:43,189 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d05ed5d2be3e1cf2a5acf4/lava_callback.json.gz 2026-04-04 01:03:43,260 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:43,267 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a2d2be3e1cf2a5b0d9/lava_callback.json.gz DEBUG: Setup failed for 69d05ed5d2be3e1cf2a5acf4. Transitting job node result: incomplete -> fail 2026-04-04 01:03:43,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:43,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:43,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ed5d2be3e1cf2a5acf4 HTTP/1.1" 200 None 2026-04-04 01:03:43,336 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:43,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a2d2be3e1cf2a5b0d9 HTTP/1.1" 200 None 2026-04-04 01:03:43,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:43,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 01:03:43,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:43,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:43,366 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:43,412 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ed5d2be3e1cf2a5acf4 HTTP/1.1" 200 None 2026-04-04 01:03:43,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061a2d2be3e1cf2a5b0d9 HTTP/1.1" 200 None 2026-04-04 01:03:43,452 - lava_callback - INFO - Completed processing callback for node 69d061a2d2be3e1cf2a5b0d9 2026-04-04 01:03:43,530 - lava_callback - INFO - Completed processing callback for node 69d05ed5d2be3e1cf2a5acf4 INFO: 172.20.0.1:58506 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:03:47,021 - lava_callback - INFO - Received callback for node 69d061a8d2be3e1cf2a5b0e8 INFO: 172.20.0.1:58514 - "POST /node/69d061a8d2be3e1cf2a5b0e8 HTTP/1.0" 202 Accepted 2026-04-04 01:03:47,031 - lava_callback - INFO - Starting async job submit for node 69d061a8d2be3e1cf2a5b0e8 2026-04-04 01:03:47,078 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:47,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a8d2be3e1cf2a5b0e8 HTTP/1.1" 200 None 2026-04-04 01:03:47,106 - lava_callback - DEBUG - Processing node 69d061a8d2be3e1cf2a5b0e8: baseline-nfs-arm64 2026-04-04 01:03:47,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:47,897 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:47,900 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a8d2be3e1cf2a5b0e8/log.txt.gz 2026-04-04 01:03:48,224 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:48,636 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:48,639 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a8d2be3e1cf2a5b0e8/lava-logs.yaml 2026-04-04 01:03:48,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:03:49,127 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:03:49,129 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a8d2be3e1cf2a5b0e8/lava_callback.json.gz 2026-04-04 01:03:49,170 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:49,191 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a8d2be3e1cf2a5b0e8 HTTP/1.1" 200 None 2026-04-04 01:03:49,193 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:49,212 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:03:49,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:49,251 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061a8d2be3e1cf2a5b0e8 HTTP/1.1" 200 None 2026-04-04 01:03:49,252 - lava_callback - INFO - Completed processing callback for node 69d061a8d2be3e1cf2a5b0e8 2026-04-04 01:03:53,231 - lava_callback - INFO - Received callback for node 69cdc07223eb053795d42343 INFO: 172.20.0.1:57976 - "POST /node/69cdc07223eb053795d42343 HTTP/1.0" 202 Accepted 2026-04-04 01:03:53,237 - lava_callback - INFO - Starting async job submit for node 69cdc07223eb053795d42343 2026-04-04 01:03:53,238 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:53,259 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc07223eb053795d42343 HTTP/1.1" 200 None 2026-04-04 01:03:53,260 - lava_callback - DEBUG - Processing node 69cdc07223eb053795d42343: baseline-arm-baylibre 2026-04-04 01:03:53,260 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc07223eb053795d42343 2026-04-04 01:03:53,261 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:53,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc07223eb053795d42343 HTTP/1.1" 200 None 2026-04-04 01:03:53,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:53,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 01:03:53,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:03:53,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc07223eb053795d42343 HTTP/1.1" 200 None 2026-04-04 01:03:53,347 - lava_callback - ERROR - Error processing callback for node 69cdc07223eb053795d42343: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57986 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:04:03,057 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:03,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 15 INFO: 172.20.0.1:47412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47416 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:04:08,020 - lava_callback - INFO - Received callback for node 69d06196d2be3e1cf2a5b08c INFO: 172.20.0.1:47430 - "POST /node/69d06196d2be3e1cf2a5b08c HTTP/1.0" 202 Accepted 2026-04-04 01:04:08,030 - lava_callback - INFO - Starting async job submit for node 69d06196d2be3e1cf2a5b08c 2026-04-04 01:04:08,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:08,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06196d2be3e1cf2a5b08c HTTP/1.1" 200 None 2026-04-04 01:04:08,102 - lava_callback - DEBUG - Processing node 69d06196d2be3e1cf2a5b08c: baseline-arm64 2026-04-04 01:04:08,796 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:04:09,211 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:04:09,213 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06196d2be3e1cf2a5b08c/log.txt.gz 2026-04-04 01:04:09,618 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:04:10,038 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:04:10,041 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06196d2be3e1cf2a5b08c/lava-logs.yaml 2026-04-04 01:04:10,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:04:10,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:04:10,321 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06196d2be3e1cf2a5b08c/lava_callback.json.gz 2026-04-04 01:04:10,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:10,429 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06196d2be3e1cf2a5b08c HTTP/1.1" 200 None 2026-04-04 01:04:10,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:10,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:04:10,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:10,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06196d2be3e1cf2a5b08c HTTP/1.1" 200 None 2026-04-04 01:04:10,553 - lava_callback - INFO - Completed processing callback for node 69d06196d2be3e1cf2a5b08c 2026-04-04 01:04:10,906 - lava_callback - INFO - Received callback for node 69d061add2be3e1cf2a5b0f4 INFO: 172.20.0.1:51004 - "POST /node/69d061add2be3e1cf2a5b0f4 HTTP/1.0" 202 Accepted 2026-04-04 01:04:10,914 - lava_callback - INFO - Starting async job submit for node 69d061add2be3e1cf2a5b0f4 2026-04-04 01:04:10,938 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:10,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061add2be3e1cf2a5b0f4 HTTP/1.1" 200 None 2026-04-04 01:04:10,952 - lava_callback - DEBUG - Processing node 69d061add2be3e1cf2a5b0f4: baseline-nfs-arm64 2026-04-04 01:04:11,552 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:04:12,067 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:04:12,069 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061add2be3e1cf2a5b0f4/log.txt.gz 2026-04-04 01:04:12,390 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:04:12,762 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:04:12,764 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061add2be3e1cf2a5b0f4/lava-logs.yaml 2026-04-04 01:04:12,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:04:13,268 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:04:13,270 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061add2be3e1cf2a5b0f4/lava_callback.json.gz 2026-04-04 01:04:13,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:13,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061add2be3e1cf2a5b0f4 HTTP/1.1" 200 None 2026-04-04 01:04:13,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:13,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:04:13,372 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:13,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061add2be3e1cf2a5b0f4 HTTP/1.1" 200 None 2026-04-04 01:04:13,398 - lava_callback - INFO - Completed processing callback for node 69d061add2be3e1cf2a5b0f4 INFO: 172.20.0.1:51016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56822 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:04:28,583 - lava_callback - INFO - Received callback for node 69d0619fd2be3e1cf2a5b0d3 INFO: 172.20.0.1:56830 - "POST /node/69d0619fd2be3e1cf2a5b0d3 HTTP/1.0" 202 Accepted 2026-04-04 01:04:28,595 - lava_callback - INFO - Starting async job submit for node 69d0619fd2be3e1cf2a5b0d3 2026-04-04 01:04:28,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:28,655 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0619fd2be3e1cf2a5b0d3 HTTP/1.1" 200 None 2026-04-04 01:04:28,657 - lava_callback - DEBUG - Processing node 69d0619fd2be3e1cf2a5b0d3: baseline-nfs-arm64 2026-04-04 01:04:29,239 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:04:29,434 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:04:29,436 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d0619fd2be3e1cf2a5b0d3/log.txt.gz 2026-04-04 01:04:29,740 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:04:30,093 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:04:30,096 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d0619fd2be3e1cf2a5b0d3/lava-logs.yaml 2026-04-04 01:04:30,105 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:04:30,656 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:04:30,659 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d0619fd2be3e1cf2a5b0d3/lava_callback.json.gz 2026-04-04 01:04:30,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:30,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0619fd2be3e1cf2a5b0d3 HTTP/1.1" 200 None 2026-04-04 01:04:30,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:30,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:04:30,749 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:30,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0619fd2be3e1cf2a5b0d3 HTTP/1.1" 200 None 2026-04-04 01:04:30,783 - lava_callback - INFO - Completed processing callback for node 69d0619fd2be3e1cf2a5b0d3 2026-04-04 01:04:33,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:33,111 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 01:04:33,265 - lava_callback - INFO - Received callback for node 69cdc09523eb053795d42355 INFO: 172.20.0.1:58924 - "POST /node/69cdc09523eb053795d42355 HTTP/1.0" 202 Accepted 2026-04-04 01:04:33,279 - lava_callback - INFO - Starting async job submit for node 69cdc09523eb053795d42355 2026-04-04 01:04:33,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:33,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc09523eb053795d42355 HTTP/1.1" 200 None 2026-04-04 01:04:33,306 - lava_callback - DEBUG - Processing node 69cdc09523eb053795d42355: baseline-arm-baylibre 2026-04-04 01:04:33,306 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc09523eb053795d42355 2026-04-04 01:04:33,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:33,333 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc09523eb053795d42355 HTTP/1.1" 200 None 2026-04-04 01:04:33,335 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:33,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 01:04:33,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:04:33,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc09523eb053795d42355 HTTP/1.1" 200 None 2026-04-04 01:04:33,429 - lava_callback - ERROR - Error processing callback for node 69cdc09523eb053795d42355: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57772 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:05:13,270 - lava_callback - INFO - Received callback for node 69cdc0b923eb053795d42361 INFO: 172.20.0.1:35760 - "POST /node/69cdc0b923eb053795d42361 HTTP/1.0" 202 Accepted 2026-04-04 01:05:13,288 - lava_callback - INFO - Starting async job submit for node 69cdc0b923eb053795d42361 2026-04-04 01:05:13,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:05:13,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc0b923eb053795d42361 HTTP/1.1" 200 None 2026-04-04 01:05:13,323 - lava_callback - DEBUG - Processing node 69cdc0b923eb053795d42361: baseline-arm-baylibre 2026-04-04 01:05:13,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc0b923eb053795d42361 2026-04-04 01:05:13,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:05:13,345 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc0b923eb053795d42361 HTTP/1.1" 200 None 2026-04-04 01:05:13,348 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:05:13,377 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 01:05:13,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:05:13,416 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc0b923eb053795d42361 HTTP/1.1" 200 None 2026-04-04 01:05:13,427 - lava_callback - ERROR - Error processing callback for node 69cdc0b923eb053795d42361: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46710 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:05:50,603 - lava_callback - INFO - Received callback for node 69d06191d2be3e1cf2a5b080 INFO: 172.20.0.1:39238 - "POST /node/69d06191d2be3e1cf2a5b080 HTTP/1.0" 202 Accepted 2026-04-04 01:05:50,610 - lava_callback - INFO - Starting async job submit for node 69d06191d2be3e1cf2a5b080 2026-04-04 01:05:50,637 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:05:50,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06191d2be3e1cf2a5b080 HTTP/1.1" 200 None 2026-04-04 01:05:50,653 - lava_callback - DEBUG - Processing node 69d06191d2be3e1cf2a5b080: baseline-arm64 2026-04-04 01:05:51,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:05:51,155 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:05:51,157 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06191d2be3e1cf2a5b080/log.txt.gz 2026-04-04 01:05:51,397 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:05:51,615 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:05:51,617 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06191d2be3e1cf2a5b080/lava-logs.yaml 2026-04-04 01:05:51,626 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:05:51,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:05:51,755 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06191d2be3e1cf2a5b080/lava_callback.json.gz 2026-04-04 01:05:51,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:05:51,796 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06191d2be3e1cf2a5b080 HTTP/1.1" 200 None 2026-04-04 01:05:51,798 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:05:51,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:05:51,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:05:51,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06191d2be3e1cf2a5b080 HTTP/1.1" 200 None 2026-04-04 01:05:51,865 - lava_callback - INFO - Completed processing callback for node 69d06191d2be3e1cf2a5b080 INFO: 172.20.0.1:39242 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:06:01,584 - lava_callback - INFO - Received callback for node 69d063efd2be3e1cf2a5b571 INFO: 172.20.0.1:53318 - "POST /node/69d063efd2be3e1cf2a5b571 HTTP/1.0" 202 Accepted 2026-04-04 01:06:01,591 - lava_callback - INFO - Starting async job submit for node 69d063efd2be3e1cf2a5b571 2026-04-04 01:06:01,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:01,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d063efd2be3e1cf2a5b571 HTTP/1.1" 200 None 2026-04-04 01:06:01,609 - lava_callback - DEBUG - Processing node 69d063efd2be3e1cf2a5b571: baseline-nfs-arm64 2026-04-04 01:06:01,611 - lava_callback - WARNING - Failed to upload log 2026-04-04 01:06:01,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:06:01,752 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:06:01,754 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d063efd2be3e1cf2a5b571/lava-logs.yaml 2026-04-04 01:06:01,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:06:01,898 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:06:01,900 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d063efd2be3e1cf2a5b571/lava_callback.json.gz 2026-04-04 01:06:01,902 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:01,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d063efd2be3e1cf2a5b571 HTTP/1.1" 200 None 2026-04-04 01:06:01,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:01,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:06:01,935 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:01,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d063efd2be3e1cf2a5b571 HTTP/1.1" 200 None 2026-04-04 01:06:01,966 - lava_callback - INFO - Completed processing callback for node 69d063efd2be3e1cf2a5b571 2026-04-04 01:06:03,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:03,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53332 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:06:07,371 - lava_callback - INFO - Received callback for node 69d061a1d2be3e1cf2a5b0d6 INFO: 172.20.0.1:53338 - "POST /node/69d061a1d2be3e1cf2a5b0d6 HTTP/1.0" 202 Accepted 2026-04-04 01:06:07,381 - lava_callback - INFO - Starting async job submit for node 69d061a1d2be3e1cf2a5b0d6 2026-04-04 01:06:07,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:07,427 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a1d2be3e1cf2a5b0d6 HTTP/1.1" 200 None 2026-04-04 01:06:07,429 - lava_callback - DEBUG - Processing node 69d061a1d2be3e1cf2a5b0d6: baseline-nfs-arm64 2026-04-04 01:06:07,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:06:08,075 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:06:08,077 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a1d2be3e1cf2a5b0d6/log.txt.gz 2026-04-04 01:06:08,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:06:08,597 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:06:08,599 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a1d2be3e1cf2a5b0d6/lava-logs.yaml 2026-04-04 01:06:08,611 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:06:08,767 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:06:08,769 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a1d2be3e1cf2a5b0d6/lava_callback.json.gz 2026-04-04 01:06:08,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:08,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a1d2be3e1cf2a5b0d6 HTTP/1.1" 200 None 2026-04-04 01:06:08,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:08,818 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:06:08,819 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:08,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061a1d2be3e1cf2a5b0d6 HTTP/1.1" 200 None 2026-04-04 01:06:08,849 - lava_callback - INFO - Completed processing callback for node 69d061a1d2be3e1cf2a5b0d6 2026-04-04 01:06:14,819 - lava_callback - INFO - Received callback for node 69cdc10423eb053795d4237b INFO: 172.20.0.1:57580 - "POST /node/69cdc10423eb053795d4237b HTTP/1.0" 202 Accepted 2026-04-04 01:06:14,828 - lava_callback - INFO - Starting async job submit for node 69cdc10423eb053795d4237b 2026-04-04 01:06:14,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:14,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc10423eb053795d4237b HTTP/1.1" 200 None 2026-04-04 01:06:14,851 - lava_callback - DEBUG - Processing node 69cdc10423eb053795d4237b: baseline-arm-clabbe 2026-04-04 01:06:14,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc10423eb053795d4237b 2026-04-04 01:06:14,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:14,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc10423eb053795d4237b HTTP/1.1" 200 None 2026-04-04 01:06:14,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:14,896 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 01:06:14,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:14,921 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc10423eb053795d4237b HTTP/1.1" 200 None 2026-04-04 01:06:14,922 - lava_callback - ERROR - Error processing callback for node 69cdc10423eb053795d4237b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43930 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:06:33,144 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:33,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:06:53,410 - lava_callback - INFO - Received callback for node 69cdc12623eb053795d42385 INFO: 172.20.0.1:38382 - "POST /node/69cdc12623eb053795d42385 HTTP/1.0" 202 Accepted 2026-04-04 01:06:53,416 - lava_callback - INFO - Starting async job submit for node 69cdc12623eb053795d42385 2026-04-04 01:06:53,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:53,432 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc12623eb053795d42385 HTTP/1.1" 200 None 2026-04-04 01:06:53,433 - lava_callback - DEBUG - Processing node 69cdc12623eb053795d42385: baseline-arm-baylibre 2026-04-04 01:06:53,433 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc12623eb053795d42385 2026-04-04 01:06:53,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:53,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc12623eb053795d42385 HTTP/1.1" 200 None 2026-04-04 01:06:53,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:53,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 01:06:53,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:06:53,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc12623eb053795d42385 HTTP/1.1" 200 None 2026-04-04 01:06:53,518 - lava_callback - ERROR - Error processing callback for node 69cdc12623eb053795d42385: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44922 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54186 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:07:33,414 - lava_callback - INFO - Received callback for node 69cdc14a23eb053795d42391 INFO: 172.20.0.1:57546 - "POST /node/69cdc14a23eb053795d42391 HTTP/1.0" 202 Accepted 2026-04-04 01:07:33,419 - lava_callback - INFO - Starting async job submit for node 69cdc14a23eb053795d42391 2026-04-04 01:07:33,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:07:33,435 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc14a23eb053795d42391 HTTP/1.1" 200 None 2026-04-04 01:07:33,435 - lava_callback - DEBUG - Processing node 69cdc14a23eb053795d42391: baseline-arm-baylibre 2026-04-04 01:07:33,435 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc14a23eb053795d42391 2026-04-04 01:07:33,436 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:07:33,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc14a23eb053795d42391 HTTP/1.1" 200 None 2026-04-04 01:07:33,460 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:07:33,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 01:07:33,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:07:33,502 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc14a23eb053795d42391 HTTP/1.1" 200 None 2026-04-04 01:07:33,503 - lava_callback - ERROR - Error processing callback for node 69cdc14a23eb053795d42391: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37138 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:08:13,477 - lava_callback - INFO - Received callback for node 69cdc17423eb053795d4241f INFO: 172.20.0.1:58738 - "POST /node/69cdc17423eb053795d4241f HTTP/1.0" 202 Accepted 2026-04-04 01:08:13,482 - lava_callback - INFO - Starting async job submit for node 69cdc17423eb053795d4241f 2026-04-04 01:08:13,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:13,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17423eb053795d4241f HTTP/1.1" 200 None 2026-04-04 01:08:13,499 - lava_callback - DEBUG - Processing node 69cdc17423eb053795d4241f: baseline-arm64-baylibre 2026-04-04 01:08:13,499 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc17423eb053795d4241f 2026-04-04 01:08:13,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:13,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17423eb053795d4241f HTTP/1.1" 200 None 2026-04-04 01:08:13,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:13,527 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:13,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:13,550 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc17423eb053795d4241f HTTP/1.1" 200 None 2026-04-04 01:08:13,555 - lava_callback - ERROR - Error processing callback for node 69cdc17423eb053795d4241f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 01:08:14,279 - lava_callback - INFO - Received callback for node 69cdc17623eb053795d42427 INFO: 172.20.0.1:58752 - "POST /node/69cdc17623eb053795d42427 HTTP/1.0" 202 Accepted 2026-04-04 01:08:14,284 - lava_callback - INFO - Starting async job submit for node 69cdc17623eb053795d42427 2026-04-04 01:08:14,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:14,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17623eb053795d42427 HTTP/1.1" 200 None 2026-04-04 01:08:14,300 - lava_callback - DEBUG - Processing node 69cdc17623eb053795d42427: baseline-arm64-baylibre 2026-04-04 01:08:14,300 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc17623eb053795d42427 2026-04-04 01:08:14,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:14,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17623eb053795d42427 HTTP/1.1" 200 None 2026-04-04 01:08:14,315 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:14,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:14,329 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:14,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc17623eb053795d42427 HTTP/1.1" 200 None 2026-04-04 01:08:14,355 - lava_callback - ERROR - Error processing callback for node 69cdc17623eb053795d42427: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 01:08:14,923 - lava_callback - INFO - Received callback for node 69cdc17723eb053795d4242a INFO: 172.20.0.1:58756 - "POST /node/69cdc17723eb053795d4242a HTTP/1.0" 202 Accepted 2026-04-04 01:08:14,932 - lava_callback - INFO - Starting async job submit for node 69cdc17723eb053795d4242a 2026-04-04 01:08:14,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:14,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17723eb053795d4242a HTTP/1.1" 200 None 2026-04-04 01:08:14,960 - lava_callback - DEBUG - Processing node 69cdc17723eb053795d4242a: baseline-arm64-baylibre 2026-04-04 01:08:14,960 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc17723eb053795d4242a 2026-04-04 01:08:14,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:14,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17723eb053795d4242a HTTP/1.1" 200 None 2026-04-04 01:08:14,977 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:14,999 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:15,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:15,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc17723eb053795d4242a HTTP/1.1" 200 None 2026-04-04 01:08:15,030 - lava_callback - ERROR - Error processing callback for node 69cdc17723eb053795d4242a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58770 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-04 01:08:15,592 - lava_callback - INFO - Received callback for node 69cdc17923eb053795d42437 INFO: 172.20.0.1:58786 - "POST /node/69cdc17923eb053795d42437 HTTP/1.0" 202 Accepted 2026-04-04 01:08:15,598 - lava_callback - INFO - Starting async job submit for node 69cdc17923eb053795d42437 2026-04-04 01:08:15,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:15,623 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17923eb053795d42437 HTTP/1.1" 200 None 2026-04-04 01:08:15,624 - lava_callback - DEBUG - Processing node 69cdc17923eb053795d42437: baseline-arm64-baylibre 2026-04-04 01:08:15,625 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc17923eb053795d42437 2026-04-04 01:08:15,625 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:15,645 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17923eb053795d42437 HTTP/1.1" 200 None 2026-04-04 01:08:15,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:15,665 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:15,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:15,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc17923eb053795d42437 HTTP/1.1" 200 None 2026-04-04 01:08:15,694 - lava_callback - ERROR - Error processing callback for node 69cdc17923eb053795d42437: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 01:08:16,264 - lava_callback - INFO - Received callback for node 69cdc17b23eb053795d4243a INFO: 172.20.0.1:58796 - "POST /node/69cdc17b23eb053795d4243a HTTP/1.0" 202 Accepted 2026-04-04 01:08:16,271 - lava_callback - INFO - Starting async job submit for node 69cdc17b23eb053795d4243a 2026-04-04 01:08:16,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:16,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17b23eb053795d4243a HTTP/1.1" 200 None 2026-04-04 01:08:16,290 - lava_callback - DEBUG - Processing node 69cdc17b23eb053795d4243a: baseline-arm64-baylibre 2026-04-04 01:08:16,290 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc17b23eb053795d4243a 2026-04-04 01:08:16,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:16,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17b23eb053795d4243a HTTP/1.1" 200 None 2026-04-04 01:08:16,305 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:16,319 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:16,320 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:16,342 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc17b23eb053795d4243a HTTP/1.1" 200 None 2026-04-04 01:08:16,343 - lava_callback - ERROR - Error processing callback for node 69cdc17b23eb053795d4243a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51668 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:08:33,456 - lava_callback - INFO - Received callback for node 69cdc17c23eb053795d4243d INFO: 172.20.0.1:40810 - "POST /node/69cdc17c23eb053795d4243d HTTP/1.0" 202 Accepted 2026-04-04 01:08:33,465 - lava_callback - INFO - Starting async job submit for node 69cdc17c23eb053795d4243d 2026-04-04 01:08:33,467 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:33,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17c23eb053795d4243d HTTP/1.1" 200 None 2026-04-04 01:08:33,512 - lava_callback - DEBUG - Processing node 69cdc17c23eb053795d4243d: baseline-arm64-baylibre 2026-04-04 01:08:33,513 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc17c23eb053795d4243d 2026-04-04 01:08:33,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:33,534 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc17c23eb053795d4243d HTTP/1.1" 200 None 2026-04-04 01:08:33,537 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:33,558 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:33,560 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:33,586 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc17c23eb053795d4243d HTTP/1.1" 200 None 2026-04-04 01:08:33,587 - lava_callback - ERROR - Error processing callback for node 69cdc17c23eb053795d4243d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 01:08:34,961 - lava_callback - INFO - Received callback for node 69cdc18c23eb053795d4247a INFO: 172.20.0.1:40816 - "POST /node/69cdc18c23eb053795d4247a HTTP/1.0" 202 Accepted 2026-04-04 01:08:34,966 - lava_callback - INFO - Starting async job submit for node 69cdc18c23eb053795d4247a 2026-04-04 01:08:34,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:35,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18c23eb053795d4247a HTTP/1.1" 200 None 2026-04-04 01:08:35,002 - lava_callback - DEBUG - Processing node 69cdc18c23eb053795d4247a: baseline-arm64-clabbe 2026-04-04 01:08:35,002 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc18c23eb053795d4247a 2026-04-04 01:08:35,002 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:35,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18c23eb053795d4247a HTTP/1.1" 200 None 2026-04-04 01:08:35,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:35,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:35,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:35,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18c23eb053795d4247a HTTP/1.1" 200 None 2026-04-04 01:08:35,061 - lava_callback - ERROR - Error processing callback for node 69cdc18c23eb053795d4247a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 01:08:35,646 - lava_callback - INFO - Received callback for node 69cdc18e23eb053795d4247d INFO: 172.20.0.1:40824 - "POST /node/69cdc18e23eb053795d4247d HTTP/1.0" 202 Accepted 2026-04-04 01:08:35,652 - lava_callback - INFO - Starting async job submit for node 69cdc18e23eb053795d4247d 2026-04-04 01:08:35,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:35,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18e23eb053795d4247d HTTP/1.1" 200 None 2026-04-04 01:08:35,667 - lava_callback - DEBUG - Processing node 69cdc18e23eb053795d4247d: baseline-arm64-clabbe 2026-04-04 01:08:35,667 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc18e23eb053795d4247d 2026-04-04 01:08:35,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:35,680 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18e23eb053795d4247d HTTP/1.1" 200 None 2026-04-04 01:08:35,682 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:35,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:35,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:35,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18e23eb053795d4247d HTTP/1.1" 200 None 2026-04-04 01:08:35,717 - lava_callback - ERROR - Error processing callback for node 69cdc18e23eb053795d4247d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 01:08:36,337 - lava_callback - INFO - Received callback for node 69cdc18e23eb053795d42480 INFO: 172.20.0.1:40826 - "POST /node/69cdc18e23eb053795d42480 HTTP/1.0" 202 Accepted 2026-04-04 01:08:36,343 - lava_callback - INFO - Starting async job submit for node 69cdc18e23eb053795d42480 2026-04-04 01:08:36,344 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:36,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18e23eb053795d42480 HTTP/1.1" 200 None 2026-04-04 01:08:36,363 - lava_callback - DEBUG - Processing node 69cdc18e23eb053795d42480: baseline-arm64-clabbe 2026-04-04 01:08:36,364 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc18e23eb053795d42480 2026-04-04 01:08:36,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:36,381 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18e23eb053795d42480 HTTP/1.1" 200 None 2026-04-04 01:08:36,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:36,397 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:36,398 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:36,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18e23eb053795d42480 HTTP/1.1" 200 None 2026-04-04 01:08:36,426 - lava_callback - ERROR - Error processing callback for node 69cdc18e23eb053795d42480: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40830 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:08:37,028 - lava_callback - INFO - Received callback for node 69cdc18f23eb053795d42483 INFO: 172.20.0.1:40844 - "POST /node/69cdc18f23eb053795d42483 HTTP/1.0" 202 Accepted 2026-04-04 01:08:37,034 - lava_callback - INFO - Starting async job submit for node 69cdc18f23eb053795d42483 2026-04-04 01:08:37,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:37,050 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18f23eb053795d42483 HTTP/1.1" 200 None 2026-04-04 01:08:37,051 - lava_callback - DEBUG - Processing node 69cdc18f23eb053795d42483: baseline-arm64-clabbe 2026-04-04 01:08:37,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc18f23eb053795d42483 2026-04-04 01:08:37,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:37,077 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc18f23eb053795d42483 HTTP/1.1" 200 None 2026-04-04 01:08:37,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:37,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:37,099 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:37,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc18f23eb053795d42483 HTTP/1.1" 200 None 2026-04-04 01:08:37,136 - lava_callback - ERROR - Error processing callback for node 69cdc18f23eb053795d42483: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 01:08:37,701 - lava_callback - INFO - Received callback for node 69cdc19123eb053795d42489 INFO: 172.20.0.1:40860 - "POST /node/69cdc19123eb053795d42489 HTTP/1.0" 202 Accepted 2026-04-04 01:08:37,710 - lava_callback - INFO - Starting async job submit for node 69cdc19123eb053795d42489 2026-04-04 01:08:37,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:37,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19123eb053795d42489 HTTP/1.1" 200 None 2026-04-04 01:08:37,728 - lava_callback - DEBUG - Processing node 69cdc19123eb053795d42489: baseline-arm64-clabbe 2026-04-04 01:08:37,729 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc19123eb053795d42489 2026-04-04 01:08:37,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:37,748 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19123eb053795d42489 HTTP/1.1" 200 None 2026-04-04 01:08:37,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:37,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:37,767 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:37,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc19123eb053795d42489 HTTP/1.1" 200 None 2026-04-04 01:08:37,796 - lava_callback - ERROR - Error processing callback for node 69cdc19123eb053795d42489: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 01:08:39,974 - lava_callback - INFO - Received callback for node 69d06460d2be3e1cf2a5b5e6 INFO: 172.20.0.1:58662 - "POST /node/69d06460d2be3e1cf2a5b5e6 HTTP/1.0" 202 Accepted 2026-04-04 01:08:39,980 - lava_callback - INFO - Starting async job submit for node 69d06460d2be3e1cf2a5b5e6 2026-04-04 01:08:39,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:40,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06460d2be3e1cf2a5b5e6 HTTP/1.1" 200 None 2026-04-04 01:08:40,018 - lava_callback - DEBUG - Processing node 69d06460d2be3e1cf2a5b5e6: baseline-arm64-pengutronix 2026-04-04 01:08:40,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:08:40,326 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:08:40,328 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d06460d2be3e1cf2a5b5e6/log.txt.gz 2026-04-04 01:08:40,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:08:40,620 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:08:40,622 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d06460d2be3e1cf2a5b5e6/lava-logs.yaml 2026-04-04 01:08:40,628 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:08:40,794 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:08:40,795 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-pengutronix-69d06460d2be3e1cf2a5b5e6/lava_callback.json.gz 2026-04-04 01:08:40,823 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:40,844 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06460d2be3e1cf2a5b5e6 HTTP/1.1" 200 None 2026-04-04 01:08:40,846 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:40,859 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:08:40,861 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:40,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06460d2be3e1cf2a5b5e6 HTTP/1.1" 200 None 2026-04-04 01:08:40,888 - lava_callback - INFO - Completed processing callback for node 69d06460d2be3e1cf2a5b5e6 2026-04-04 01:08:45,436 - lava_callback - INFO - Received callback for node 69d06486d2be3e1cf2a5b619 INFO: 172.20.0.1:58674 - "POST /node/69d06486d2be3e1cf2a5b619 HTTP/1.0" 202 Accepted 2026-04-04 01:08:45,443 - lava_callback - INFO - Starting async job submit for node 69d06486d2be3e1cf2a5b619 2026-04-04 01:08:45,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:45,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06486d2be3e1cf2a5b619 HTTP/1.1" 200 None 2026-04-04 01:08:45,462 - lava_callback - DEBUG - Processing node 69d06486d2be3e1cf2a5b619: baseline-arm 2026-04-04 01:08:45,467 - lava_callback - WARNING - Failed to upload log 2026-04-04 01:08:45,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:08:45,635 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:08:45,637 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d06486d2be3e1cf2a5b619/lava-logs.yaml 2026-04-04 01:08:45,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:08:45,782 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:08:45,784 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d06486d2be3e1cf2a5b619/lava_callback.json.gz 2026-04-04 01:08:45,788 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:45,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06486d2be3e1cf2a5b619 HTTP/1.1" 200 None 2026-04-04 01:08:45,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:45,823 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 01:08:45,825 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:45,847 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06486d2be3e1cf2a5b619 HTTP/1.1" 200 None 2026-04-04 01:08:45,852 - lava_callback - INFO - Completed processing callback for node 69d06486d2be3e1cf2a5b619 INFO: 172.20.0.1:58688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58692 - "GET /node/6927915b97dccbdf516a6be1%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-04 01:08:55,070 - lava_callback - INFO - Received callback for node 69cdc19223eb053795d4248c INFO: 172.20.0.1:60234 - "POST /node/69cdc19223eb053795d4248c HTTP/1.0" 202 Accepted 2026-04-04 01:08:55,076 - lava_callback - INFO - Starting async job submit for node 69cdc19223eb053795d4248c 2026-04-04 01:08:55,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:55,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19223eb053795d4248c HTTP/1.1" 200 None 2026-04-04 01:08:55,092 - lava_callback - DEBUG - Processing node 69cdc19223eb053795d4248c: baseline-arm64-clabbe 2026-04-04 01:08:55,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc19223eb053795d4248c 2026-04-04 01:08:55,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:55,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19223eb053795d4248c HTTP/1.1" 200 None 2026-04-04 01:08:55,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:55,120 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 01:08:55,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:55,143 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc19223eb053795d4248c HTTP/1.1" 200 None 2026-04-04 01:08:55,144 - lava_callback - ERROR - Error processing callback for node 69cdc19223eb053795d4248c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 01:08:55,749 - lava_callback - INFO - Received callback for node 69cdc19523eb053795d42491 INFO: 172.20.0.1:60250 - "POST /node/69cdc19523eb053795d42491 HTTP/1.0" 202 Accepted 2026-04-04 01:08:55,758 - lava_callback - INFO - Starting async job submit for node 69cdc19523eb053795d42491 2026-04-04 01:08:55,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:55,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19523eb053795d42491 HTTP/1.1" 200 None 2026-04-04 01:08:55,776 - lava_callback - DEBUG - Processing node 69cdc19523eb053795d42491: baseline-arm-clabbe 2026-04-04 01:08:55,777 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc19523eb053795d42491 2026-04-04 01:08:55,777 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:55,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc19523eb053795d42491 HTTP/1.1" 200 None 2026-04-04 01:08:55,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:55,814 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 01:08:55,816 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:08:55,838 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc19523eb053795d42491 HTTP/1.1" 200 None 2026-04-04 01:08:55,840 - lava_callback - ERROR - Error processing callback for node 69cdc19523eb053795d42491: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60256 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:09:03,172 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:09:03,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:56282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56284 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:09:14,952 - lava_callback - INFO - Received callback for node 69cdc1b823eb053795d424c8 INFO: 172.20.0.1:33942 - "POST /node/69cdc1b823eb053795d424c8 HTTP/1.0" 202 Accepted 2026-04-04 01:09:14,957 - lava_callback - INFO - Starting async job submit for node 69cdc1b823eb053795d424c8 2026-04-04 01:09:14,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:09:14,976 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc1b823eb053795d424c8 HTTP/1.1" 200 None 2026-04-04 01:09:14,977 - lava_callback - DEBUG - Processing node 69cdc1b823eb053795d424c8: baseline-arm-clabbe 2026-04-04 01:09:14,977 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc1b823eb053795d424c8 2026-04-04 01:09:14,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:09:14,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc1b823eb053795d424c8 HTTP/1.1" 200 None 2026-04-04 01:09:14,993 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:09:15,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 01:09:15,008 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:09:15,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc1b823eb053795d424c8 HTTP/1.1" 200 None 2026-04-04 01:09:15,034 - lava_callback - ERROR - Error processing callback for node 69cdc1b823eb053795d424c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33958 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55176 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:09:55,004 - lava_callback - INFO - Received callback for node 69cdc1dd23eb053795d424f1 INFO: 172.20.0.1:36878 - "POST /node/69cdc1dd23eb053795d424f1 HTTP/1.0" 202 Accepted 2026-04-04 01:09:55,010 - lava_callback - INFO - Starting async job submit for node 69cdc1dd23eb053795d424f1 2026-04-04 01:09:55,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:09:55,027 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc1dd23eb053795d424f1 HTTP/1.1" 200 None 2026-04-04 01:09:55,028 - lava_callback - DEBUG - Processing node 69cdc1dd23eb053795d424f1: baseline-arm-clabbe 2026-04-04 01:09:55,028 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc1dd23eb053795d424f1 2026-04-04 01:09:55,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:09:55,042 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc1dd23eb053795d424f1 HTTP/1.1" 200 None 2026-04-04 01:09:55,044 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:09:55,078 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 01:09:55,080 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:09:55,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc1dd23eb053795d424f1 HTTP/1.1" 200 None 2026-04-04 01:09:55,106 - lava_callback - ERROR - Error processing callback for node 69cdc1dd23eb053795d424f1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33108 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:10:34,960 - lava_callback - INFO - Received callback for node 69cdc20223eb053795d42517 INFO: 172.20.0.1:59098 - "POST /node/69cdc20223eb053795d42517 HTTP/1.0" 202 Accepted 2026-04-04 01:10:34,966 - lava_callback - INFO - Starting async job submit for node 69cdc20223eb053795d42517 2026-04-04 01:10:34,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:10:34,988 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc20223eb053795d42517 HTTP/1.1" 200 None 2026-04-04 01:10:34,989 - lava_callback - DEBUG - Processing node 69cdc20223eb053795d42517: baseline-arm-clabbe 2026-04-04 01:10:34,989 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc20223eb053795d42517 2026-04-04 01:10:34,990 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:10:35,002 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc20223eb053795d42517 HTTP/1.1" 200 None 2026-04-04 01:10:35,004 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:10:35,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 01:10:35,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:10:35,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc20223eb053795d42517 HTTP/1.1" 200 None 2026-04-04 01:10:35,042 - lava_callback - ERROR - Error processing callback for node 69cdc20223eb053795d42517: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59986 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:10:55,607 - lava_callback - INFO - Received callback for node 69d0619ed2be3e1cf2a5b0d0 INFO: 172.20.0.1:54488 - "POST /node/69d0619ed2be3e1cf2a5b0d0 HTTP/1.0" 202 Accepted 2026-04-04 01:10:55,614 - lava_callback - INFO - Starting async job submit for node 69d0619ed2be3e1cf2a5b0d0 2026-04-04 01:10:55,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:10:55,667 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0619ed2be3e1cf2a5b0d0 HTTP/1.1" 200 None 2026-04-04 01:10:55,668 - lava_callback - DEBUG - Processing node 69d0619ed2be3e1cf2a5b0d0: baseline-arm64 2026-04-04 01:10:56,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:10:56,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:10:56,197 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619ed2be3e1cf2a5b0d0/log.txt.gz 2026-04-04 01:10:56,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:54500 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:10:56,641 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:10:56,643 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619ed2be3e1cf2a5b0d0/lava-logs.yaml 2026-04-04 01:10:56,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:10:56,829 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:10:56,830 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0619ed2be3e1cf2a5b0d0/lava_callback.json.gz DEBUG: Setup failed for 69d0619ed2be3e1cf2a5b0d0. Transitting job node result: incomplete -> fail 2026-04-04 01:10:56,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:10:56,917 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0619ed2be3e1cf2a5b0d0 HTTP/1.1" 200 None 2026-04-04 01:10:56,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:10:56,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:10:56,933 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:10:56,955 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0619ed2be3e1cf2a5b0d0 HTTP/1.1" 200 None 2026-04-04 01:10:57,020 - lava_callback - INFO - Completed processing callback for node 69d0619ed2be3e1cf2a5b0d0 2026-04-04 01:11:03,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:11:03,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:51842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53632 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:12:38,811 - lava_callback - INFO - Received callback for node 69d05ef2d2be3e1cf2a5ad8f INFO: 172.20.0.1:53634 - "POST /node/69d05ef2d2be3e1cf2a5ad8f HTTP/1.0" 202 Accepted 2026-04-04 01:12:38,858 - lava_callback - INFO - Starting async job submit for node 69d05ef2d2be3e1cf2a5ad8f 2026-04-04 01:12:38,885 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:12:38,903 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef2d2be3e1cf2a5ad8f HTTP/1.1" 200 None 2026-04-04 01:12:38,904 - lava_callback - DEBUG - Processing node 69d05ef2d2be3e1cf2a5ad8f: ltp-sched 2026-04-04 01:12:38,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:12:39,132 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:12:39,134 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69d05ef2d2be3e1cf2a5ad8f/log.txt.gz 2026-04-04 01:12:39,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:12:39,366 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:12:39,368 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69d05ef2d2be3e1cf2a5ad8f/lava-logs.yaml 2026-04-04 01:12:39,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:12:39,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:12:39,502 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69d05ef2d2be3e1cf2a5ad8f/lava_callback.json.gz 2026-04-04 01:12:39,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:12:39,543 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d05ef2d2be3e1cf2a5ad8f HTTP/1.1" 200 None 2026-04-04 01:12:39,545 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:12:39,561 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 01:12:39,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:12:39,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d05ef2d2be3e1cf2a5ad8f HTTP/1.1" 200 None 2026-04-04 01:12:39,650 - lava_callback - INFO - Completed processing callback for node 69d05ef2d2be3e1cf2a5ad8f 2026-04-04 01:12:45,555 - lava_callback - INFO - Received callback for node 69d06199d2be3e1cf2a5b095 INFO: 172.20.0.1:40900 - "POST /node/69d06199d2be3e1cf2a5b095 HTTP/1.0" 202 Accepted 2026-04-04 01:12:45,564 - lava_callback - INFO - Starting async job submit for node 69d06199d2be3e1cf2a5b095 2026-04-04 01:12:45,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:12:45,619 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06199d2be3e1cf2a5b095 HTTP/1.1" 200 None 2026-04-04 01:12:45,620 - lava_callback - DEBUG - Processing node 69d06199d2be3e1cf2a5b095: baseline-arm64 2026-04-04 01:12:46,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:12:46,408 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:12:46,410 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06199d2be3e1cf2a5b095/log.txt.gz INFO: 172.20.0.1:40904 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:12:46,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:12:46,994 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:12:46,997 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06199d2be3e1cf2a5b095/lava-logs.yaml 2026-04-04 01:12:47,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:12:47,195 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:12:47,197 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06199d2be3e1cf2a5b095/lava_callback.json.gz 2026-04-04 01:12:47,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:12:47,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06199d2be3e1cf2a5b095 HTTP/1.1" 200 None 2026-04-04 01:12:47,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:12:47,272 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:12:47,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:12:47,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06199d2be3e1cf2a5b095 HTTP/1.1" 200 None 2026-04-04 01:12:47,381 - lava_callback - INFO - Completed processing callback for node 69d06199d2be3e1cf2a5b095 INFO: 172.20.0.1:33892 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:13:02,431 - lava_callback - INFO - Received callback for node 69d061a9d2be3e1cf2a5b0eb INFO: 172.20.0.1:40236 - "POST /node/69d061a9d2be3e1cf2a5b0eb HTTP/1.0" 202 Accepted 2026-04-04 01:13:02,437 - lava_callback - INFO - Starting async job submit for node 69d061a9d2be3e1cf2a5b0eb 2026-04-04 01:13:02,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:13:02,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a9d2be3e1cf2a5b0eb HTTP/1.1" 200 None 2026-04-04 01:13:02,479 - lava_callback - DEBUG - Processing node 69d061a9d2be3e1cf2a5b0eb: baseline-nfs-arm64 2026-04-04 01:13:03,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:13:03,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:13:03,193 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a9d2be3e1cf2a5b0eb/log.txt.gz 2026-04-04 01:13:03,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:13:03,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 01:13:03,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:13:03,758 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:13:03,760 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a9d2be3e1cf2a5b0eb/lava-logs.yaml 2026-04-04 01:13:03,773 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:13:03,940 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:13:03,942 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a9d2be3e1cf2a5b0eb/lava_callback.json.gz 2026-04-04 01:13:03,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:13:03,990 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a9d2be3e1cf2a5b0eb HTTP/1.1" 200 None 2026-04-04 01:13:03,992 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:13:04,009 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:13:04,011 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:13:04,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061a9d2be3e1cf2a5b0eb HTTP/1.1" 200 None 2026-04-04 01:13:04,093 - lava_callback - INFO - Completed processing callback for node 69d061a9d2be3e1cf2a5b0eb INFO: 172.20.0.1:40248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50512 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48572 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:13:33,333 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:13:33,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34806 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45152 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:14:53,951 - lava_callback - INFO - Received callback for node 69cdc30123eb053795d4264a INFO: 172.20.0.1:46040 - "POST /node/69cdc30123eb053795d4264a HTTP/1.0" 202 Accepted 2026-04-04 01:14:53,957 - lava_callback - INFO - Starting async job submit for node 69cdc30123eb053795d4264a 2026-04-04 01:14:53,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:14:53,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc30123eb053795d4264a HTTP/1.1" 200 None 2026-04-04 01:14:53,973 - lava_callback - DEBUG - Processing node 69cdc30123eb053795d4264a: baseline-arm64-baylibre 2026-04-04 01:14:53,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc30123eb053795d4264a 2026-04-04 01:14:53,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:14:53,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc30123eb053795d4264a HTTP/1.1" 200 None 2026-04-04 01:14:53,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:14:54,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-04 01:14:54,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:14:54,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc30123eb053795d4264a HTTP/1.1" 200 None 2026-04-04 01:14:54,036 - lava_callback - ERROR - Error processing callback for node 69cdc30123eb053795d4264a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54984 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59822 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:16:39,089 - lava_callback - INFO - Received callback for node 69d06601d2be3e1cf2a5b7a5 INFO: 172.20.0.1:59832 - "POST /node/69d06601d2be3e1cf2a5b7a5 HTTP/1.0" 202 Accepted 2026-04-04 01:16:39,095 - lava_callback - INFO - Starting async job submit for node 69d06601d2be3e1cf2a5b7a5 2026-04-04 01:16:39,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:16:39,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06601d2be3e1cf2a5b7a5 HTTP/1.1" 200 None 2026-04-04 01:16:39,138 - lava_callback - DEBUG - Processing node 69d06601d2be3e1cf2a5b7a5: baseline-arm64-broonie 2026-04-04 01:16:39,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:16:39,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:16:39,621 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d06601d2be3e1cf2a5b7a5/log.txt.gz 2026-04-04 01:16:39,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:16:40,111 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:16:40,113 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d06601d2be3e1cf2a5b7a5/lava-logs.yaml 2026-04-04 01:16:40,122 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:16:40,308 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:16:40,310 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-broonie-69d06601d2be3e1cf2a5b7a5/lava_callback.json.gz 2026-04-04 01:16:40,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:16:40,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06601d2be3e1cf2a5b7a5 HTTP/1.1" 200 None 2026-04-04 01:16:40,349 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:16:40,363 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:16:40,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:16:40,388 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06601d2be3e1cf2a5b7a5 HTTP/1.1" 200 None 2026-04-04 01:16:40,390 - lava_callback - INFO - Completed processing callback for node 69d06601d2be3e1cf2a5b7a5 INFO: 172.20.0.1:39042 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:16:53,305 - lava_callback - INFO - Received callback for node 69d0663dd2be3e1cf2a5b7c4 INFO: 172.20.0.1:56566 - "POST /node/69d0663dd2be3e1cf2a5b7c4 HTTP/1.0" 202 Accepted 2026-04-04 01:16:53,312 - lava_callback - INFO - Starting async job submit for node 69d0663dd2be3e1cf2a5b7c4 2026-04-04 01:16:53,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:16:53,407 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0663dd2be3e1cf2a5b7c4 HTTP/1.1" 200 None 2026-04-04 01:16:53,408 - lava_callback - DEBUG - Processing node 69d0663dd2be3e1cf2a5b7c4: baseline-arm64 2026-04-04 01:16:53,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:16:53,873 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:16:53,875 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0663dd2be3e1cf2a5b7c4/log.txt.gz 2026-04-04 01:16:54,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:16:54,406 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:16:54,408 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0663dd2be3e1cf2a5b7c4/lava-logs.yaml 2026-04-04 01:16:54,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:16:54,570 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:16:54,573 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0663dd2be3e1cf2a5b7c4/lava_callback.json.gz 2026-04-04 01:16:54,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:16:54,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0663dd2be3e1cf2a5b7c4 HTTP/1.1" 200 None 2026-04-04 01:16:54,616 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:16:54,638 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:16:54,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:16:54,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0663dd2be3e1cf2a5b7c4 HTTP/1.1" 200 None 2026-04-04 01:16:54,669 - lava_callback - INFO - Completed processing callback for node 69d0663dd2be3e1cf2a5b7c4 INFO: 172.20.0.1:56570 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:17:03,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:17:03,371 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51996 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:18:34,084 - lava_callback - INFO - Received callback for node 69cdc3e423eb053795d42684 INFO: 172.20.0.1:36282 - "POST /node/69cdc3e423eb053795d42684 HTTP/1.0" 202 Accepted 2026-04-04 01:18:34,093 - lava_callback - INFO - Starting async job submit for node 69cdc3e423eb053795d42684 2026-04-04 01:18:34,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:18:34,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc3e423eb053795d42684 HTTP/1.1" 200 None 2026-04-04 01:18:34,114 - lava_callback - DEBUG - Processing node 69cdc3e423eb053795d42684: baseline-arm64-baylibre 2026-04-04 01:18:34,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc3e423eb053795d42684 2026-04-04 01:18:34,115 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:18:34,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc3e423eb053795d42684 HTTP/1.1" 200 None 2026-04-04 01:18:34,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:18:34,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-04 01:18:34,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:18:34,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc3e423eb053795d42684 HTTP/1.1" 200 None 2026-04-04 01:18:34,180 - lava_callback - ERROR - Error processing callback for node 69cdc3e423eb053795d42684: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59002 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33738 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:19:54,390 - lava_callback - INFO - Received callback for node 69d066c0d2be3e1cf2a5b839 INFO: 172.20.0.1:42438 - "POST /node/69d066c0d2be3e1cf2a5b839 HTTP/1.0" 202 Accepted 2026-04-04 01:19:54,401 - lava_callback - INFO - Starting async job submit for node 69d066c0d2be3e1cf2a5b839 2026-04-04 01:19:54,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:19:54,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d066c0d2be3e1cf2a5b839 HTTP/1.1" 200 None 2026-04-04 01:19:54,480 - lava_callback - DEBUG - Processing node 69d066c0d2be3e1cf2a5b839: baseline-arm64 2026-04-04 01:19:55,159 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:19:55,474 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:19:55,476 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d066c0d2be3e1cf2a5b839/log.txt.gz 2026-04-04 01:19:55,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:19:56,242 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:19:56,244 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d066c0d2be3e1cf2a5b839/lava-logs.yaml 2026-04-04 01:19:56,254 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:19:56,402 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:19:56,405 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d066c0d2be3e1cf2a5b839/lava_callback.json.gz 2026-04-04 01:19:56,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:19:56,486 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d066c0d2be3e1cf2a5b839 HTTP/1.1" 200 None 2026-04-04 01:19:56,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:19:56,519 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:19:56,522 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:42446 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:19:56,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d066c0d2be3e1cf2a5b839 HTTP/1.1" 200 None 2026-04-04 01:19:56,665 - lava_callback - INFO - Completed processing callback for node 69d066c0d2be3e1cf2a5b839 2026-04-04 01:20:03,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:20:03,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:34252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43590 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59516 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48048 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:21:56,484 - lava_callback - INFO - Received callback for node 69d0678cd2be3e1cf2a5b913 INFO: 172.20.0.1:49198 - "POST /node/69d0678cd2be3e1cf2a5b913 HTTP/1.0" 202 Accepted 2026-04-04 01:21:56,490 - lava_callback - INFO - Starting async job submit for node 69d0678cd2be3e1cf2a5b913 2026-04-04 01:21:56,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:21:56,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0678cd2be3e1cf2a5b913 HTTP/1.1" 200 None 2026-04-04 01:21:56,513 - lava_callback - DEBUG - Processing node 69d0678cd2be3e1cf2a5b913: baseline-nfs-arm64 INFO: 172.20.0.1:49210 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:21:56,515 - lava_callback - WARNING - Failed to upload log 2026-04-04 01:21:56,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:21:56,666 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:21:56,667 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d0678cd2be3e1cf2a5b913/lava-logs.yaml 2026-04-04 01:21:56,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:21:56,806 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:21:56,808 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d0678cd2be3e1cf2a5b913/lava_callback.json.gz 2026-04-04 01:21:56,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:21:56,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0678cd2be3e1cf2a5b913 HTTP/1.1" 200 None 2026-04-04 01:21:56,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:21:56,855 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:21:56,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:21:56,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0678cd2be3e1cf2a5b913 HTTP/1.1" 200 None 2026-04-04 01:21:56,886 - lava_callback - INFO - Completed processing callback for node 69d0678cd2be3e1cf2a5b913 2026-04-04 01:22:03,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:22:03,428 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 01:22:05,530 - lava_callback - INFO - Received callback for node 69d061a3d2be3e1cf2a5b0dc INFO: 172.20.0.1:42518 - "POST /node/69d061a3d2be3e1cf2a5b0dc HTTP/1.0" 202 Accepted 2026-04-04 01:22:05,539 - lava_callback - INFO - Starting async job submit for node 69d061a3d2be3e1cf2a5b0dc 2026-04-04 01:22:05,587 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:22:05,602 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a3d2be3e1cf2a5b0dc HTTP/1.1" 200 None 2026-04-04 01:22:05,603 - lava_callback - DEBUG - Processing node 69d061a3d2be3e1cf2a5b0dc: baseline-nfs-arm64 INFO: 172.20.0.1:42526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42540 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:22:06,860 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:22:07,040 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:22:07,041 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a3d2be3e1cf2a5b0dc/log.txt.gz 2026-04-04 01:22:07,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:22:07,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:22:08,003 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a3d2be3e1cf2a5b0dc/lava-logs.yaml 2026-04-04 01:22:08,024 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:22:08,275 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:22:08,277 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d061a3d2be3e1cf2a5b0dc/lava_callback.json.gz DEBUG: Setup failed for 69d061a3d2be3e1cf2a5b0dc. Transitting job node result: incomplete -> fail 2026-04-04 01:22:08,369 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:22:08,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d061a3d2be3e1cf2a5b0dc HTTP/1.1" 200 None 2026-04-04 01:22:08,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:22:08,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:22:08,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:22:08,443 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d061a3d2be3e1cf2a5b0dc HTTP/1.1" 200 None 2026-04-04 01:22:08,575 - lava_callback - INFO - Completed processing callback for node 69d061a3d2be3e1cf2a5b0dc INFO: 172.20.0.1:35304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51638 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:22:33,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:22:33,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 01:22:34,434 - lava_callback - INFO - Received callback for node 69cdc4c523eb053795d426a8 INFO: 172.20.0.1:37146 - "POST /node/69cdc4c523eb053795d426a8 HTTP/1.0" 202 Accepted 2026-04-04 01:22:34,440 - lava_callback - INFO - Starting async job submit for node 69cdc4c523eb053795d426a8 2026-04-04 01:22:34,441 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:22:34,456 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc4c523eb053795d426a8 HTTP/1.1" 200 None 2026-04-04 01:22:34,457 - lava_callback - DEBUG - Processing node 69cdc4c523eb053795d426a8: baseline-arm64-baylibre 2026-04-04 01:22:34,457 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc4c523eb053795d426a8 2026-04-04 01:22:34,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:22:34,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc4c523eb053795d426a8 HTTP/1.1" 200 None 2026-04-04 01:22:34,480 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:22:34,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-04 01:22:34,501 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:22:34,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc4c523eb053795d426a8 HTTP/1.1" 200 None 2026-04-04 01:22:34,526 - lava_callback - ERROR - Error processing callback for node 69cdc4c523eb053795d426a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48990 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:23:09,168 - lava_callback - INFO - Received callback for node 69d06544d2be3e1cf2a5b70f INFO: 172.20.0.1:49004 - "POST /node/69d06544d2be3e1cf2a5b70f HTTP/1.0" 202 Accepted 2026-04-04 01:23:09,175 - lava_callback - INFO - Starting async job submit for node 69d06544d2be3e1cf2a5b70f 2026-04-04 01:23:09,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:23:09,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06544d2be3e1cf2a5b70f HTTP/1.1" 200 None 2026-04-04 01:23:09,219 - lava_callback - DEBUG - Processing node 69d06544d2be3e1cf2a5b70f: baseline-arm64 2026-04-04 01:23:09,353 - lava_callback - WARNING - Failed to upload log 2026-04-04 01:23:09,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:23:09,659 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:23:09,661 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06544d2be3e1cf2a5b70f/lava-logs.yaml 2026-04-04 01:23:09,666 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:23:09,825 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:23:09,827 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06544d2be3e1cf2a5b70f/lava_callback.json.gz 2026-04-04 01:23:09,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:23:09,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06544d2be3e1cf2a5b70f HTTP/1.1" 200 None 2026-04-04 01:23:09,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:23:09,915 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:23:09,916 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:23:09,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06544d2be3e1cf2a5b70f HTTP/1.1" 200 None 2026-04-04 01:23:10,028 - lava_callback - INFO - Completed processing callback for node 69d06544d2be3e1cf2a5b70f INFO: 172.20.0.1:42800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45482 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:23:33,451 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:23:33,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53168 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:24:54,424 - lava_callback - INFO - Received callback for node 69cdc55623eb053795d42a4e INFO: 172.20.0.1:33636 - "POST /node/69cdc55623eb053795d42a4e HTTP/1.0" 202 Accepted 2026-04-04 01:24:54,430 - lava_callback - INFO - Starting async job submit for node 69cdc55623eb053795d42a4e 2026-04-04 01:24:54,430 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:24:54,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc55623eb053795d42a4e HTTP/1.1" 200 None 2026-04-04 01:24:54,446 - lava_callback - DEBUG - Processing node 69cdc55623eb053795d42a4e: baseline-arm64-baylibre 2026-04-04 01:24:54,446 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc55623eb053795d42a4e 2026-04-04 01:24:54,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:24:54,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc55623eb053795d42a4e HTTP/1.1" 200 None 2026-04-04 01:24:54,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:24:54,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 01:24:54,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:24:54,500 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc55623eb053795d42a4e HTTP/1.1" 200 None 2026-04-04 01:24:54,501 - lava_callback - ERROR - Error processing callback for node 69cdc55623eb053795d42a4e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58332 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58344 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:25:19,025 - lava_callback - INFO - Received callback for node 69d065b2d2be3e1cf2a5b744 INFO: 172.20.0.1:58356 - "POST /node/69d065b2d2be3e1cf2a5b744 HTTP/1.0" 202 Accepted 2026-04-04 01:25:19,032 - lava_callback - INFO - Starting async job submit for node 69d065b2d2be3e1cf2a5b744 2026-04-04 01:25:19,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:25:19,084 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d065b2d2be3e1cf2a5b744 HTTP/1.1" 200 None 2026-04-04 01:25:19,085 - lava_callback - DEBUG - Processing node 69d065b2d2be3e1cf2a5b744: baseline-arm64 2026-04-04 01:25:19,239 - lava_callback - WARNING - Failed to upload log 2026-04-04 01:25:19,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:25:19,525 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:25:19,527 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d065b2d2be3e1cf2a5b744/lava-logs.yaml 2026-04-04 01:25:19,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:25:19,674 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:25:19,676 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d065b2d2be3e1cf2a5b744/lava_callback.json.gz 2026-04-04 01:25:19,726 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:25:19,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d065b2d2be3e1cf2a5b744 HTTP/1.1" 200 None 2026-04-04 01:25:19,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:25:19,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:25:19,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:25:19,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d065b2d2be3e1cf2a5b744 HTTP/1.1" 200 None 2026-04-04 01:25:19,862 - lava_callback - INFO - Completed processing callback for node 69d065b2d2be3e1cf2a5b744 INFO: 172.20.0.1:46972 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:25:33,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:25:33,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:37670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59908 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56244 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:27:14,565 - lava_callback - INFO - Received callback for node 69cdc5eb23eb053795d42a6a INFO: 172.20.0.1:54488 - "POST /node/69cdc5eb23eb053795d42a6a HTTP/1.0" 202 Accepted 2026-04-04 01:27:14,572 - lava_callback - INFO - Starting async job submit for node 69cdc5eb23eb053795d42a6a 2026-04-04 01:27:14,573 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:27:14,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc5eb23eb053795d42a6a HTTP/1.1" 200 None 2026-04-04 01:27:14,588 - lava_callback - DEBUG - Processing node 69cdc5eb23eb053795d42a6a: baseline-arm64-baylibre 2026-04-04 01:27:14,588 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc5eb23eb053795d42a6a 2026-04-04 01:27:14,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:27:14,601 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc5eb23eb053795d42a6a HTTP/1.1" 200 None 2026-04-04 01:27:14,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:27:14,632 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 01:27:14,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:27:14,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc5eb23eb053795d42a6a HTTP/1.1" 200 None 2026-04-04 01:27:14,664 - lava_callback - ERROR - Error processing callback for node 69cdc5eb23eb053795d42a6a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35904 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:28:11,832 - lava_callback - INFO - Received callback for node 69d06527d2be3e1cf2a5b6e1 INFO: 172.20.0.1:56188 - "POST /node/69d06527d2be3e1cf2a5b6e1 HTTP/1.0" 202 Accepted 2026-04-04 01:28:11,843 - lava_callback - INFO - Starting async job submit for node 69d06527d2be3e1cf2a5b6e1 2026-04-04 01:28:11,915 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:28:11,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06527d2be3e1cf2a5b6e1 HTTP/1.1" 200 None 2026-04-04 01:28:11,935 - lava_callback - DEBUG - Processing node 69d06527d2be3e1cf2a5b6e1: baseline-arm 2026-04-04 01:28:12,652 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:28:12,815 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:28:12,817 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d06527d2be3e1cf2a5b6e1/log.txt.gz 2026-04-04 01:28:13,203 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:28:13,440 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:28:13,442 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d06527d2be3e1cf2a5b6e1/lava-logs.yaml 2026-04-04 01:28:13,457 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:28:13,618 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:28:13,619 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d06527d2be3e1cf2a5b6e1/lava_callback.json.gz DEBUG: Setup failed for 69d06527d2be3e1cf2a5b6e1. Transitting job node result: incomplete -> fail 2026-04-04 01:28:13,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:28:13,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06527d2be3e1cf2a5b6e1 HTTP/1.1" 200 None 2026-04-04 01:28:13,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:28:13,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 01:28:13,762 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:28:13,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06527d2be3e1cf2a5b6e1 HTTP/1.1" 200 None 2026-04-04 01:28:13,894 - lava_callback - INFO - Completed processing callback for node 69d06527d2be3e1cf2a5b6e1 INFO: 172.20.0.1:56190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38294 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:28:33,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:28:33,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 2026-04-04 01:28:34,659 - lava_callback - INFO - Received callback for node 69cdc63923eb053795d42a77 INFO: 172.20.0.1:47514 - "POST /node/69cdc63923eb053795d42a77 HTTP/1.0" 202 Accepted 2026-04-04 01:28:34,666 - lava_callback - INFO - Starting async job submit for node 69cdc63923eb053795d42a77 2026-04-04 01:28:34,667 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:28:34,690 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc63923eb053795d42a77 HTTP/1.1" 200 None 2026-04-04 01:28:34,692 - lava_callback - DEBUG - Processing node 69cdc63923eb053795d42a77: baseline-arm64-baylibre 2026-04-04 01:28:34,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc63923eb053795d42a77 2026-04-04 01:28:34,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:28:34,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc63923eb053795d42a77 HTTP/1.1" 200 None 2026-04-04 01:28:34,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:28:34,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 01:28:34,724 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:28:34,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc63923eb053795d42a77 HTTP/1.1" 200 None 2026-04-04 01:28:34,746 - lava_callback - ERROR - Error processing callback for node 69cdc63923eb053795d42a77: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41444 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42334 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:29:54,755 - lava_callback - INFO - Received callback for node 69cdc68623eb053795d42a8c INFO: 172.20.0.1:59942 - "POST /node/69cdc68623eb053795d42a8c HTTP/1.0" 202 Accepted 2026-04-04 01:29:54,760 - lava_callback - INFO - Starting async job submit for node 69cdc68623eb053795d42a8c 2026-04-04 01:29:54,761 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:29:54,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc68623eb053795d42a8c HTTP/1.1" 200 None 2026-04-04 01:29:54,778 - lava_callback - DEBUG - Processing node 69cdc68623eb053795d42a8c: baseline-arm64-baylibre 2026-04-04 01:29:54,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc68623eb053795d42a8c 2026-04-04 01:29:54,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:29:54,794 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc68623eb053795d42a8c HTTP/1.1" 200 None 2026-04-04 01:29:54,795 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:29:54,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 01:29:54,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:29:54,841 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc68623eb053795d42a8c HTTP/1.1" 200 None 2026-04-04 01:29:54,843 - lava_callback - ERROR - Error processing callback for node 69cdc68623eb053795d42a8c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50568 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52264 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:30:58,837 - lava_callback - INFO - Received callback for node 69d065afd2be3e1cf2a5b741 INFO: 172.20.0.1:52266 - "POST /node/69d065afd2be3e1cf2a5b741 HTTP/1.0" 202 Accepted 2026-04-04 01:30:58,844 - lava_callback - INFO - Starting async job submit for node 69d065afd2be3e1cf2a5b741 2026-04-04 01:30:58,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:30:58,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d065afd2be3e1cf2a5b741 HTTP/1.1" 200 None 2026-04-04 01:30:58,923 - lava_callback - DEBUG - Processing node 69d065afd2be3e1cf2a5b741: ltp-smoke 2026-04-04 01:30:59,404 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:30:59,548 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:30:59,550 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-smoke-69d065afd2be3e1cf2a5b741/log.txt.gz 2026-04-04 01:30:59,854 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:31:00,102 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:31:00,105 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-smoke-69d065afd2be3e1cf2a5b741/lava-logs.yaml 2026-04-04 01:31:00,118 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:31:00,302 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:31:00,304 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-smoke-69d065afd2be3e1cf2a5b741/lava_callback.json.gz 2026-04-04 01:31:00,466 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:31:00,487 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d065afd2be3e1cf2a5b741 HTTP/1.1" 200 None 2026-04-04 01:31:00,489 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:31:00,511 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cfd2be3e1cf2a5a7f5 HTTP/1.1" 200 None 2026-04-04 01:31:00,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:31:00,537 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d065afd2be3e1cf2a5b741 HTTP/1.1" 200 None 2026-04-04 01:31:00,675 - lava_callback - INFO - Completed processing callback for node 69d065afd2be3e1cf2a5b741 2026-04-04 01:31:03,517 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:31:03,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:47460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53270 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-04 01:31:15,876 - lava_callback - INFO - Received callback for node 69cdc6d923eb053795d42a9f INFO: 172.20.0.1:53284 - "POST /node/69cdc6d923eb053795d42a9f HTTP/1.0" 202 Accepted 2026-04-04 01:31:15,881 - lava_callback - INFO - Starting async job submit for node 69cdc6d923eb053795d42a9f 2026-04-04 01:31:15,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:31:15,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc6d923eb053795d42a9f HTTP/1.1" 200 None 2026-04-04 01:31:15,897 - lava_callback - DEBUG - Processing node 69cdc6d923eb053795d42a9f: baseline-arm64-clabbe 2026-04-04 01:31:15,898 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc6d923eb053795d42a9f 2026-04-04 01:31:15,898 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:31:15,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc6d923eb053795d42a9f HTTP/1.1" 200 None 2026-04-04 01:31:15,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:31:15,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 01:31:15,941 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:31:15,960 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc6d923eb053795d42a9f HTTP/1.1" 200 None 2026-04-04 01:31:15,961 - lava_callback - ERROR - Error processing callback for node 69cdc6d923eb053795d42a9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46140 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55098 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:32:35,910 - lava_callback - INFO - Received callback for node 69cdc72523eb053795d42ab0 INFO: 172.20.0.1:54018 - "POST /node/69cdc72523eb053795d42ab0 HTTP/1.0" 202 Accepted 2026-04-04 01:32:35,916 - lava_callback - INFO - Starting async job submit for node 69cdc72523eb053795d42ab0 2026-04-04 01:32:35,918 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:32:35,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc72523eb053795d42ab0 HTTP/1.1" 200 None 2026-04-04 01:32:35,933 - lava_callback - DEBUG - Processing node 69cdc72523eb053795d42ab0: baseline-arm64-clabbe 2026-04-04 01:32:35,934 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc72523eb053795d42ab0 2026-04-04 01:32:35,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:32:35,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc72523eb053795d42ab0 HTTP/1.1" 200 None 2026-04-04 01:32:35,958 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:32:35,971 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 01:32:35,973 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:32:36,004 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc72523eb053795d42ab0 HTTP/1.1" 200 None 2026-04-04 01:32:36,006 - lava_callback - ERROR - Error processing callback for node 69cdc72523eb053795d42ab0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37510 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37536 - "GET /node/69bd9b7b0e93e231472418e4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41246 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:33:55,962 - lava_callback - INFO - Received callback for node 69cdc77123eb053795d42ab7 INFO: 172.20.0.1:39026 - "POST /node/69cdc77123eb053795d42ab7 HTTP/1.0" 202 Accepted 2026-04-04 01:33:55,967 - lava_callback - INFO - Starting async job submit for node 69cdc77123eb053795d42ab7 2026-04-04 01:33:55,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:33:55,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc77123eb053795d42ab7 HTTP/1.1" 200 None 2026-04-04 01:33:55,994 - lava_callback - DEBUG - Processing node 69cdc77123eb053795d42ab7: baseline-arm64-clabbe 2026-04-04 01:33:55,994 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc77123eb053795d42ab7 2026-04-04 01:33:55,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:33:56,015 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc77123eb053795d42ab7 HTTP/1.1" 200 None 2026-04-04 01:33:56,017 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:33:56,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 01:33:56,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:33:56,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc77123eb053795d42ab7 HTTP/1.1" 200 None 2026-04-04 01:33:56,084 - lava_callback - ERROR - Error processing callback for node 69cdc77123eb053795d42ab7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60734 - "GET /node/69bd8e630e93e23147241599%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:60750 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48904 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:34:56,010 - lava_callback - INFO - Received callback for node 69cdc7bd23eb053795d42aca INFO: 172.20.0.1:42300 - "POST /node/69cdc7bd23eb053795d42aca HTTP/1.0" 202 Accepted 2026-04-04 01:34:56,018 - lava_callback - INFO - Starting async job submit for node 69cdc7bd23eb053795d42aca 2026-04-04 01:34:56,019 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:34:56,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc7bd23eb053795d42aca HTTP/1.1" 200 None 2026-04-04 01:34:56,041 - lava_callback - DEBUG - Processing node 69cdc7bd23eb053795d42aca: baseline-arm64-clabbe 2026-04-04 01:34:56,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc7bd23eb053795d42aca 2026-04-04 01:34:56,042 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:34:56,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc7bd23eb053795d42aca HTTP/1.1" 200 None 2026-04-04 01:34:56,066 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:34:56,086 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 01:34:56,088 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:34:56,127 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc7bd23eb053795d42aca HTTP/1.1" 200 None 2026-04-04 01:34:56,128 - lava_callback - ERROR - Error processing callback for node 69cdc7bd23eb053795d42aca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50392 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47152 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-04 01:36:16,025 - lava_callback - INFO - Received callback for node 69cdc80a23eb053795d42ad0 INFO: 172.20.0.1:47154 - "POST /node/69cdc80a23eb053795d42ad0 HTTP/1.0" 202 Accepted 2026-04-04 01:36:16,032 - lava_callback - INFO - Starting async job submit for node 69cdc80a23eb053795d42ad0 2026-04-04 01:36:16,034 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:36:16,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc80a23eb053795d42ad0 HTTP/1.1" 200 None 2026-04-04 01:36:16,053 - lava_callback - DEBUG - Processing node 69cdc80a23eb053795d42ad0: baseline-arm64-clabbe 2026-04-04 01:36:16,053 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc80a23eb053795d42ad0 2026-04-04 01:36:16,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:36:16,071 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc80a23eb053795d42ad0 HTTP/1.1" 200 None 2026-04-04 01:36:16,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:36:16,087 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 01:36:16,089 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:36:16,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc80a23eb053795d42ad0 HTTP/1.1" 200 None 2026-04-04 01:36:16,111 - lava_callback - ERROR - Error processing callback for node 69cdc80a23eb053795d42ad0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57364 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:36:30,623 - lava_callback - INFO - Received callback for node 69d06685d2be3e1cf2a5b7e6 INFO: 172.20.0.1:49786 - "POST /node/69d06685d2be3e1cf2a5b7e6 HTTP/1.0" 202 Accepted 2026-04-04 01:36:30,673 - lava_callback - INFO - Starting async job submit for node 69d06685d2be3e1cf2a5b7e6 2026-04-04 01:36:30,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:36:30,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06685d2be3e1cf2a5b7e6 HTTP/1.1" 200 None 2026-04-04 01:36:30,728 - lava_callback - DEBUG - Processing node 69d06685d2be3e1cf2a5b7e6: baseline-arm-pengutronix 2026-04-04 01:36:30,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:36:30,999 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:36:31,001 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d06685d2be3e1cf2a5b7e6/log.txt.gz 2026-04-04 01:36:31,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:36:31,262 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:36:31,264 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d06685d2be3e1cf2a5b7e6/lava-logs.yaml 2026-04-04 01:36:31,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:36:31,436 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:36:31,438 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d06685d2be3e1cf2a5b7e6/lava_callback.json.gz DEBUG: Setup failed for 69d06685d2be3e1cf2a5b7e6. Transitting job node result: incomplete -> fail 2026-04-04 01:36:31,487 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:36:31,507 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06685d2be3e1cf2a5b7e6 HTTP/1.1" 200 None 2026-04-04 01:36:31,509 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:36:31,528 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 01:36:31,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:36:31,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06685d2be3e1cf2a5b7e6 HTTP/1.1" 200 None 2026-04-04 01:36:31,597 - lava_callback - INFO - Completed processing callback for node 69d06685d2be3e1cf2a5b7e6 2026-04-04 01:36:33,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:36:33,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59658 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:37:36,089 - lava_callback - INFO - Received callback for node 69cdc85623eb053795d42ade INFO: 172.20.0.1:43846 - "POST /node/69cdc85623eb053795d42ade HTTP/1.0" 202 Accepted 2026-04-04 01:37:36,094 - lava_callback - INFO - Starting async job submit for node 69cdc85623eb053795d42ade 2026-04-04 01:37:36,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:37:36,110 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc85623eb053795d42ade HTTP/1.1" 200 None 2026-04-04 01:37:36,110 - lava_callback - DEBUG - Processing node 69cdc85623eb053795d42ade: baseline-arm64-clabbe 2026-04-04 01:37:36,111 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc85623eb053795d42ade 2026-04-04 01:37:36,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:37:36,134 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc85623eb053795d42ade HTTP/1.1" 200 None 2026-04-04 01:37:36,136 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:37:36,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 01:37:36,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:37:36,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc85623eb053795d42ade HTTP/1.1" 200 None 2026-04-04 01:37:36,171 - lava_callback - ERROR - Error processing callback for node 69cdc85623eb053795d42ade: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54160 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33964 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55998 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-04 01:41:15,399 - lava_callback - INFO - Received callback for node 69cdc93023eb053795d42b06 INFO: 172.20.0.1:56010 - "POST /node/69cdc93023eb053795d42b06 HTTP/1.0" 202 Accepted 2026-04-04 01:41:15,405 - lava_callback - INFO - Starting async job submit for node 69cdc93023eb053795d42b06 2026-04-04 01:41:15,406 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:41:15,420 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc93023eb053795d42b06 HTTP/1.1" 200 None 2026-04-04 01:41:15,421 - lava_callback - DEBUG - Processing node 69cdc93023eb053795d42b06: baseline-arm64-baylibre 2026-04-04 01:41:15,422 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdc93023eb053795d42b06 2026-04-04 01:41:15,422 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:41:15,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdc93023eb053795d42b06 HTTP/1.1" 200 None 2026-04-04 01:41:15,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:41:15,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-04 01:41:15,453 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:41:15,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdc93023eb053795d42b06 HTTP/1.1" 200 None 2026-04-04 01:41:15,478 - lava_callback - ERROR - Error processing callback for node 69cdc93023eb053795d42b06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41374 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52810 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-04 01:45:16,497 - lava_callback - INFO - Received callback for node 69cdca1723eb053795d42b36 INFO: 172.20.0.1:52814 - "POST /node/69cdca1723eb053795d42b36 HTTP/1.0" 202 Accepted 2026-04-04 01:45:16,503 - lava_callback - INFO - Starting async job submit for node 69cdca1723eb053795d42b36 2026-04-04 01:45:16,504 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:52830 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:45:16,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdca1723eb053795d42b36 HTTP/1.1" 200 None 2026-04-04 01:45:16,526 - lava_callback - DEBUG - Processing node 69cdca1723eb053795d42b36: baseline-arm64-clabbe 2026-04-04 01:45:16,526 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdca1723eb053795d42b36 2026-04-04 01:45:16,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:45:16,542 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdca1723eb053795d42b36 HTTP/1.1" 200 None 2026-04-04 01:45:16,544 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:45:16,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-04 01:45:16,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:45:16,607 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdca1723eb053795d42b36 HTTP/1.1" 200 None 2026-04-04 01:45:16,608 - lava_callback - ERROR - Error processing callback for node 69cdca1723eb053795d42b36: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 01:45:21,688 - lava_callback - INFO - Received callback for node 69d06c3dd2be3e1cf2a5bd07 INFO: 172.20.0.1:41066 - "POST /node/69d06c3dd2be3e1cf2a5bd07 HTTP/1.0" 202 Accepted 2026-04-04 01:45:21,694 - lava_callback - INFO - Starting async job submit for node 69d06c3dd2be3e1cf2a5bd07 2026-04-04 01:45:21,720 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:45:21,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06c3dd2be3e1cf2a5bd07 HTTP/1.1" 200 None 2026-04-04 01:45:21,737 - lava_callback - DEBUG - Processing node 69d06c3dd2be3e1cf2a5bd07: ltp-sched 2026-04-04 01:45:22,079 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:45:22,204 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:45:22,206 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/ltp-sched-69d06c3dd2be3e1cf2a5bd07/log.txt.gz 2026-04-04 01:45:22,450 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:45:22,685 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:45:22,687 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/ltp-sched-69d06c3dd2be3e1cf2a5bd07/lava-logs.yaml 2026-04-04 01:45:22,696 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:45:22,906 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:45:22,908 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/ltp-sched-69d06c3dd2be3e1cf2a5bd07/lava_callback.json.gz 2026-04-04 01:45:22,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:45:22,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06c3dd2be3e1cf2a5bd07 HTTP/1.1" 200 None 2026-04-04 01:45:22,951 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:45:22,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 01:45:22,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:45:22,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06c3dd2be3e1cf2a5bd07 HTTP/1.1" 200 None 2026-04-04 01:45:22,996 - lava_callback - INFO - Completed processing callback for node 69d06c3dd2be3e1cf2a5bd07 INFO: 172.20.0.1:41074 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:45:33,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:45:33,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48722 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:46:10,056 - lava_callback - INFO - Received callback for node 69d06d11d2be3e1cf2a5bd9d INFO: 172.20.0.1:37652 - "POST /node/69d06d11d2be3e1cf2a5bd9d HTTP/1.0" 202 Accepted 2026-04-04 01:46:10,063 - lava_callback - INFO - Starting async job submit for node 69d06d11d2be3e1cf2a5bd9d 2026-04-04 01:46:10,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:46:10,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06d11d2be3e1cf2a5bd9d HTTP/1.1" 200 None 2026-04-04 01:46:10,118 - lava_callback - DEBUG - Processing node 69d06d11d2be3e1cf2a5bd9d: baseline-arm64 2026-04-04 01:46:10,595 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:46:10,761 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:46:10,763 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06d11d2be3e1cf2a5bd9d/log.txt.gz 2026-04-04 01:46:11,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:46:11,325 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:46:11,328 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06d11d2be3e1cf2a5bd9d/lava-logs.yaml 2026-04-04 01:46:11,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:46:11,504 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:46:11,506 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06d11d2be3e1cf2a5bd9d/lava_callback.json.gz 2026-04-04 01:46:11,551 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:46:11,565 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06d11d2be3e1cf2a5bd9d HTTP/1.1" 200 None 2026-04-04 01:46:11,567 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:46:11,585 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:46:11,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:46:11,610 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06d11d2be3e1cf2a5bd9d HTTP/1.1" 200 None 2026-04-04 01:46:11,656 - lava_callback - INFO - Completed processing callback for node 69d06d11d2be3e1cf2a5bd9d INFO: 172.20.0.1:37656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37670 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:46:23,046 - lava_callback - INFO - Received callback for node 69d06d5cd2be3e1cf2a5bde2 INFO: 172.20.0.1:49362 - "POST /node/69d06d5cd2be3e1cf2a5bde2 HTTP/1.0" 202 Accepted 2026-04-04 01:46:23,052 - lava_callback - INFO - Starting async job submit for node 69d06d5cd2be3e1cf2a5bde2 2026-04-04 01:46:23,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:46:23,069 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06d5cd2be3e1cf2a5bde2 HTTP/1.1" 200 None 2026-04-04 01:46:23,069 - lava_callback - DEBUG - Processing node 69d06d5cd2be3e1cf2a5bde2: baseline-nfs-arm64 2026-04-04 01:46:23,071 - lava_callback - WARNING - Failed to upload log 2026-04-04 01:46:23,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:46:23,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:46:23,192 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d06d5cd2be3e1cf2a5bde2/lava-logs.yaml 2026-04-04 01:46:23,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:46:23,318 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:46:23,320 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d06d5cd2be3e1cf2a5bde2/lava_callback.json.gz 2026-04-04 01:46:23,322 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:46:23,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06d5cd2be3e1cf2a5bde2 HTTP/1.1" 200 None 2026-04-04 01:46:23,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:46:23,350 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:46:23,352 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:46:23,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06d5cd2be3e1cf2a5bde2 HTTP/1.1" 200 None 2026-04-04 01:46:23,375 - lava_callback - INFO - Completed processing callback for node 69d06d5cd2be3e1cf2a5bde2 INFO: 172.20.0.1:49370 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:46:33,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:46:33,599 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:33440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43416 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:47:21,083 - lava_callback - INFO - Received callback for node 69d06bead2be3e1cf2a5bc7c INFO: 172.20.0.1:35068 - "POST /node/69d06bead2be3e1cf2a5bc7c HTTP/1.0" 202 Accepted 2026-04-04 01:47:21,091 - lava_callback - INFO - Starting async job submit for node 69d06bead2be3e1cf2a5bc7c 2026-04-04 01:47:21,128 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:47:21,144 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06bead2be3e1cf2a5bc7c HTTP/1.1" 200 None 2026-04-04 01:47:21,144 - lava_callback - DEBUG - Processing node 69d06bead2be3e1cf2a5bc7c: baseline-arm64 2026-04-04 01:47:21,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:47:21,670 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:47:21,672 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06bead2be3e1cf2a5bc7c/log.txt.gz 2026-04-04 01:47:21,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:47:22,090 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:47:22,092 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06bead2be3e1cf2a5bc7c/lava-logs.yaml 2026-04-04 01:47:22,102 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:47:22,271 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:47:22,273 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06bead2be3e1cf2a5bc7c/lava_callback.json.gz DEBUG: Setup failed for 69d06bead2be3e1cf2a5bc7c. Transitting job node result: incomplete -> fail 2026-04-04 01:47:22,357 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:47:22,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06bead2be3e1cf2a5bc7c HTTP/1.1" 200 None 2026-04-04 01:47:22,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:47:22,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:47:22,393 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:47:22,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06bead2be3e1cf2a5bc7c HTTP/1.1" 200 None 2026-04-04 01:47:22,507 - lava_callback - INFO - Completed processing callback for node 69d06bead2be3e1cf2a5bc7c INFO: 172.20.0.1:35080 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:47:33,602 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:47:33,625 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43932 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:47:36,647 - lava_callback - INFO - Received callback for node 69cdcab223eb053795d42b4a INFO: 172.20.0.1:43940 - "POST /node/69cdcab223eb053795d42b4a HTTP/1.0" 202 Accepted 2026-04-04 01:47:36,653 - lava_callback - INFO - Starting async job submit for node 69cdcab223eb053795d42b4a 2026-04-04 01:47:36,654 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:47:36,676 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcab223eb053795d42b4a HTTP/1.1" 200 None 2026-04-04 01:47:36,677 - lava_callback - DEBUG - Processing node 69cdcab223eb053795d42b4a: baseline-arm64-clabbe 2026-04-04 01:47:36,677 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdcab223eb053795d42b4a 2026-04-04 01:47:36,678 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:47:36,701 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcab223eb053795d42b4a HTTP/1.1" 200 None 2026-04-04 01:47:36,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:47:36,716 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-04 01:47:36,718 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:47:36,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcab223eb053795d42b4a HTTP/1.1" 200 None 2026-04-04 01:47:36,751 - lava_callback - ERROR - Error processing callback for node 69cdcab223eb053795d42b4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34350 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:48:56,721 - lava_callback - INFO - Received callback for node 69cdcb0423eb053795d42b5b INFO: 172.20.0.1:34358 - "POST /node/69cdcb0423eb053795d42b5b HTTP/1.0" 202 Accepted 2026-04-04 01:48:56,728 - lava_callback - INFO - Starting async job submit for node 69cdcb0423eb053795d42b5b 2026-04-04 01:48:56,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:48:56,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcb0423eb053795d42b5b HTTP/1.1" 200 None 2026-04-04 01:48:56,750 - lava_callback - DEBUG - Processing node 69cdcb0423eb053795d42b5b: baseline-arm64-clabbe 2026-04-04 01:48:56,750 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdcb0423eb053795d42b5b 2026-04-04 01:48:56,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:48:56,764 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcb0423eb053795d42b5b HTTP/1.1" 200 None 2026-04-04 01:48:56,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:48:56,779 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-04 01:48:56,781 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:48:56,808 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcb0423eb053795d42b5b HTTP/1.1" 200 None 2026-04-04 01:48:56,810 - lava_callback - ERROR - Error processing callback for node 69cdcb0423eb053795d42b5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33778 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:49:07,817 - lava_callback - INFO - Received callback for node 69d06dddd2be3e1cf2a5be64 INFO: 172.20.0.1:33790 - "POST /node/69d06dddd2be3e1cf2a5be64 HTTP/1.0" 202 Accepted 2026-04-04 01:49:07,824 - lava_callback - INFO - Starting async job submit for node 69d06dddd2be3e1cf2a5be64 2026-04-04 01:49:07,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:49:07,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06dddd2be3e1cf2a5be64 HTTP/1.1" 200 None 2026-04-04 01:49:07,871 - lava_callback - DEBUG - Processing node 69d06dddd2be3e1cf2a5be64: baseline-arm64 2026-04-04 01:49:08,024 - lava_callback - WARNING - Failed to upload log 2026-04-04 01:49:08,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:49:08,278 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:49:08,280 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06dddd2be3e1cf2a5be64/lava-logs.yaml 2026-04-04 01:49:08,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:49:08,455 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:49:08,457 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06dddd2be3e1cf2a5be64/lava_callback.json.gz 2026-04-04 01:49:08,498 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:49:08,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06dddd2be3e1cf2a5be64 HTTP/1.1" 200 None 2026-04-04 01:49:08,514 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:49:08,540 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:49:08,541 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:49:08,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06dddd2be3e1cf2a5be64 HTTP/1.1" 200 None 2026-04-04 01:49:08,659 - lava_callback - INFO - Completed processing callback for node 69d06dddd2be3e1cf2a5be64 INFO: 172.20.0.1:50634 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34958 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:49:33,627 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:49:33,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:54080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34604 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55214 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44582 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:51:31,301 - lava_callback - INFO - Received callback for node 69d06c44d2be3e1cf2a5bd0a INFO: 172.20.0.1:58882 - "POST /node/69d06c44d2be3e1cf2a5bd0a HTTP/1.0" 202 Accepted 2026-04-04 01:51:31,308 - lava_callback - INFO - Starting async job submit for node 69d06c44d2be3e1cf2a5bd0a 2026-04-04 01:51:31,332 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:51:31,346 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06c44d2be3e1cf2a5bd0a HTTP/1.1" 200 None 2026-04-04 01:51:31,347 - lava_callback - DEBUG - Processing node 69d06c44d2be3e1cf2a5bd0a: baseline-arm64 2026-04-04 01:51:31,843 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:51:32,002 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:51:32,003 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06c44d2be3e1cf2a5bd0a/log.txt.gz 2026-04-04 01:51:32,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:51:32,512 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:51:32,515 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06c44d2be3e1cf2a5bd0a/lava-logs.yaml 2026-04-04 01:51:32,529 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:51:32,735 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:51:32,737 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06c44d2be3e1cf2a5bd0a/lava_callback.json.gz 2026-04-04 01:51:32,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:51:32,787 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06c44d2be3e1cf2a5bd0a HTTP/1.1" 200 None 2026-04-04 01:51:32,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:51:32,803 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:51:32,804 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:51:32,827 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06c44d2be3e1cf2a5bd0a HTTP/1.1" 200 None 2026-04-04 01:51:32,903 - lava_callback - INFO - Completed processing callback for node 69d06c44d2be3e1cf2a5bd0a 2026-04-04 01:51:33,645 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:51:33,660 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:58892 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:51:37,853 - lava_callback - INFO - Received callback for node 69d06e5ad2be3e1cf2a5bee7 INFO: 172.20.0.1:58904 - "POST /node/69d06e5ad2be3e1cf2a5bee7 HTTP/1.0" 202 Accepted 2026-04-04 01:51:37,860 - lava_callback - INFO - Starting async job submit for node 69d06e5ad2be3e1cf2a5bee7 2026-04-04 01:51:37,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:51:37,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06e5ad2be3e1cf2a5bee7 HTTP/1.1" 200 None 2026-04-04 01:51:37,908 - lava_callback - DEBUG - Processing node 69d06e5ad2be3e1cf2a5bee7: baseline-arm64 2026-04-04 01:51:38,184 - lava_callback - WARNING - Failed to upload log 2026-04-04 01:51:38,330 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:51:38,549 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:51:38,551 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06e5ad2be3e1cf2a5bee7/lava-logs.yaml 2026-04-04 01:51:38,556 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:51:38,713 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:51:38,715 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d06e5ad2be3e1cf2a5bee7/lava_callback.json.gz 2026-04-04 01:51:38,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:51:38,782 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06e5ad2be3e1cf2a5bee7 HTTP/1.1" 200 None 2026-04-04 01:51:38,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:51:38,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:51:38,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:51:38,830 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06e5ad2be3e1cf2a5bee7 HTTP/1.1" 200 None 2026-04-04 01:51:38,928 - lava_callback - INFO - Completed processing callback for node 69d06e5ad2be3e1cf2a5bee7 INFO: 172.20.0.1:57100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51108 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:52:03,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:52:03,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:53448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49638 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43852 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:52:56,832 - lava_callback - INFO - Received callback for node 69cdcbea23eb053795d42b7d INFO: 172.20.0.1:43862 - "POST /node/69cdcbea23eb053795d42b7d HTTP/1.0" 202 Accepted 2026-04-04 01:52:56,838 - lava_callback - INFO - Starting async job submit for node 69cdcbea23eb053795d42b7d 2026-04-04 01:52:56,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:52:56,862 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcbea23eb053795d42b7d HTTP/1.1" 200 None 2026-04-04 01:52:56,863 - lava_callback - DEBUG - Processing node 69cdcbea23eb053795d42b7d: baseline-arm64-clabbe 2026-04-04 01:52:56,864 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdcbea23eb053795d42b7d 2026-04-04 01:52:56,864 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:52:56,882 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcbea23eb053795d42b7d HTTP/1.1" 200 None 2026-04-04 01:52:56,883 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:52:56,899 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-04 01:52:56,901 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:52:56,926 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcbea23eb053795d42b7d HTTP/1.1" 200 None 2026-04-04 01:52:56,928 - lava_callback - ERROR - Error processing callback for node 69cdcbea23eb053795d42b7d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' 2026-04-04 01:53:06,411 - lava_callback - INFO - Received callback for node 69d06c8bd2be3e1cf2a5bd68 INFO: 172.20.0.1:46308 - "POST /node/69d06c8bd2be3e1cf2a5bd68 HTTP/1.0" 202 Accepted 2026-04-04 01:53:06,418 - lava_callback - INFO - Starting async job submit for node 69d06c8bd2be3e1cf2a5bd68 2026-04-04 01:53:06,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:53:06,472 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06c8bd2be3e1cf2a5bd68 HTTP/1.1" 200 None 2026-04-04 01:53:06,473 - lava_callback - DEBUG - Processing node 69d06c8bd2be3e1cf2a5bd68: baseline-nfs-arm64 INFO: 172.20.0.1:46320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46336 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:53:07,151 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:53:07,290 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:53:07,291 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d06c8bd2be3e1cf2a5bd68/log.txt.gz 2026-04-04 01:53:07,615 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:53:07,841 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:53:07,844 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d06c8bd2be3e1cf2a5bd68/lava-logs.yaml 2026-04-04 01:53:07,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 01:53:08,041 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 01:53:08,043 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d06c8bd2be3e1cf2a5bd68/lava_callback.json.gz 2026-04-04 01:53:08,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:53:08,101 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06c8bd2be3e1cf2a5bd68 HTTP/1.1" 200 None 2026-04-04 01:53:08,103 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:53:08,117 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 01:53:08,119 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:53:08,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06c8bd2be3e1cf2a5bd68 HTTP/1.1" 200 None 2026-04-04 01:53:08,223 - lava_callback - INFO - Completed processing callback for node 69d06c8bd2be3e1cf2a5bd68 INFO: 172.20.0.1:44148 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36188 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:53:33,687 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:53:33,703 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34672 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:54:16,909 - lava_callback - INFO - Received callback for node 69cdcc3623eb053795d42b88 INFO: 172.20.0.1:34686 - "POST /node/69cdcc3623eb053795d42b88 HTTP/1.0" 202 Accepted 2026-04-04 01:54:16,916 - lava_callback - INFO - Starting async job submit for node 69cdcc3623eb053795d42b88 2026-04-04 01:54:16,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:54:16,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcc3623eb053795d42b88 HTTP/1.1" 200 None 2026-04-04 01:54:16,933 - lava_callback - DEBUG - Processing node 69cdcc3623eb053795d42b88: baseline-arm64-clabbe 2026-04-04 01:54:16,934 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdcc3623eb053795d42b88 2026-04-04 01:54:16,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:54:16,948 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcc3623eb053795d42b88 HTTP/1.1" 200 None 2026-04-04 01:54:16,950 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:54:16,963 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-04 01:54:16,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:54:16,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcc3623eb053795d42b88 HTTP/1.1" 200 None 2026-04-04 01:54:16,998 - lava_callback - ERROR - Error processing callback for node 69cdcc3623eb053795d42b88: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51332 - "GET /node/69bda3ce0e93e23147241d3a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52218 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58076 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56990 - "GET / HTTP/1.0" 200 OK 2026-04-04 01:59:57,176 - lava_callback - INFO - Received callback for node 69cdcd8823eb053795d42bb9 INFO: 172.20.0.1:56994 - "POST /node/69cdcd8823eb053795d42bb9 HTTP/1.0" 202 Accepted 2026-04-04 01:59:57,181 - lava_callback - INFO - Starting async job submit for node 69cdcd8823eb053795d42bb9 2026-04-04 01:59:57,182 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:59:57,196 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcd8823eb053795d42bb9 HTTP/1.1" 200 None 2026-04-04 01:59:57,197 - lava_callback - DEBUG - Processing node 69cdcd8823eb053795d42bb9: baseline-arm64-clabbe 2026-04-04 01:59:57,197 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdcd8823eb053795d42bb9 2026-04-04 01:59:57,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:59:57,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcd8823eb053795d42bb9 HTTP/1.1" 200 None 2026-04-04 01:59:57,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:59:57,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c5cc3e2cb62322ad965dbc HTTP/1.1" 200 None 2026-04-04 01:59:57,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 01:59:57,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcd8823eb053795d42bb9 HTTP/1.1" 200 None 2026-04-04 01:59:57,262 - lava_callback - ERROR - Error processing callback for node 69cdcd8823eb053795d42bb9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51198 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36330 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39628 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49214 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:04:06,057 - lava_callback - INFO - Received callback for node 69d0714ed2be3e1cf2a5c15c INFO: 172.20.0.1:39200 - "POST /node/69d0714ed2be3e1cf2a5c15c HTTP/1.0" 202 Accepted 2026-04-04 02:04:06,064 - lava_callback - INFO - Starting async job submit for node 69d0714ed2be3e1cf2a5c15c 2026-04-04 02:04:06,087 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:04:06,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0714ed2be3e1cf2a5c15c HTTP/1.1" 200 None 2026-04-04 02:04:06,113 - lava_callback - DEBUG - Processing node 69d0714ed2be3e1cf2a5c15c: baseline-arm64 2026-04-04 02:04:06,543 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39206 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:04:06,708 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:04:06,710 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0714ed2be3e1cf2a5c15c/log.txt.gz 2026-04-04 02:04:06,987 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:04:07,198 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:04:07,201 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0714ed2be3e1cf2a5c15c/lava-logs.yaml 2026-04-04 02:04:07,211 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:39214 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:04:07,404 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:04:07,406 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0714ed2be3e1cf2a5c15c/lava_callback.json.gz 2026-04-04 02:04:07,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:04:07,473 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0714ed2be3e1cf2a5c15c HTTP/1.1" 200 None 2026-04-04 02:04:07,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:04:07,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:04:07,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:04:07,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0714ed2be3e1cf2a5c15c HTTP/1.1" 200 None 2026-04-04 02:04:07,567 - lava_callback - INFO - Completed processing callback for node 69d0714ed2be3e1cf2a5c15c INFO: 172.20.0.1:41342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57090 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:04:33,706 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:04:33,723 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:42330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59014 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:06:09,999 - lava_callback - INFO - Received callback for node 69d071d2d2be3e1cf2a5c1ed INFO: 172.20.0.1:40634 - "POST /node/69d071d2d2be3e1cf2a5c1ed HTTP/1.0" 202 Accepted 2026-04-04 02:06:10,009 - lava_callback - INFO - Starting async job submit for node 69d071d2d2be3e1cf2a5c1ed 2026-04-04 02:06:10,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:06:10,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d071d2d2be3e1cf2a5c1ed HTTP/1.1" 200 None 2026-04-04 02:06:10,054 - lava_callback - DEBUG - Processing node 69d071d2d2be3e1cf2a5c1ed: baseline-arm64 2026-04-04 02:06:10,185 - lava_callback - WARNING - Failed to upload log 2026-04-04 02:06:10,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:06:10,449 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:06:10,451 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d071d2d2be3e1cf2a5c1ed/lava-logs.yaml 2026-04-04 02:06:10,454 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:06:10,571 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:06:10,572 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d071d2d2be3e1cf2a5c1ed/lava_callback.json.gz 2026-04-04 02:06:10,603 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:06:10,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d071d2d2be3e1cf2a5c1ed HTTP/1.1" 200 None 2026-04-04 02:06:10,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:06:10,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:06:10,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:06:10,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d071d2d2be3e1cf2a5c1ed HTTP/1.1" 200 None 2026-04-04 02:06:10,735 - lava_callback - INFO - Completed processing callback for node 69d071d2d2be3e1cf2a5c1ed INFO: 172.20.0.1:40636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51034 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:06:33,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:06:33,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:49802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59634 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:07:02,519 - lava_callback - INFO - Received callback for node 69d06d9ed2be3e1cf2a5be3f INFO: 172.20.0.1:53850 - "POST /node/69d06d9ed2be3e1cf2a5be3f HTTP/1.0" 202 Accepted 2026-04-04 02:07:02,528 - lava_callback - INFO - Starting async job submit for node 69d06d9ed2be3e1cf2a5be3f 2026-04-04 02:07:02,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:07:02,589 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06d9ed2be3e1cf2a5be3f HTTP/1.1" 200 None 2026-04-04 02:07:02,590 - lava_callback - DEBUG - Processing node 69d06d9ed2be3e1cf2a5be3f: baseline-nfs-arm64 2026-04-04 02:07:03,834 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:07:04,021 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:07:04,023 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d06d9ed2be3e1cf2a5be3f/log.txt.gz 2026-04-04 02:07:04,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:07:04,957 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:07:04,959 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d06d9ed2be3e1cf2a5be3f/lava-logs.yaml 2026-04-04 02:07:04,981 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:07:05,162 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:07:05,164 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d06d9ed2be3e1cf2a5be3f/lava_callback.json.gz DEBUG: Setup failed for 69d06d9ed2be3e1cf2a5be3f. Transitting job node result: incomplete -> fail 2026-04-04 02:07:05,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:07:05,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06d9ed2be3e1cf2a5be3f HTTP/1.1" 200 None 2026-04-04 02:07:05,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:07:05,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:07:05,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:07:05,311 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06d9ed2be3e1cf2a5be3f HTTP/1.1" 200 None 2026-04-04 02:07:05,393 - lava_callback - INFO - Completed processing callback for node 69d06d9ed2be3e1cf2a5be3f INFO: 172.20.0.1:53856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41836 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:07:33,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:07:33,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:50990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36576 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58420 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:09:06,586 - lava_callback - INFO - Received callback for node 69d0725ed2be3e1cf2a5c229 INFO: 172.20.0.1:58424 - "POST /node/69d0725ed2be3e1cf2a5c229 HTTP/1.0" 202 Accepted 2026-04-04 02:09:06,594 - lava_callback - INFO - Starting async job submit for node 69d0725ed2be3e1cf2a5c229 2026-04-04 02:09:06,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:09:06,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0725ed2be3e1cf2a5c229 HTTP/1.1" 200 None 2026-04-04 02:09:06,675 - lava_callback - DEBUG - Processing node 69d0725ed2be3e1cf2a5c229: baseline-arm64 2026-04-04 02:09:06,889 - lava_callback - WARNING - Failed to upload log 2026-04-04 02:09:07,007 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:09:07,237 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:09:07,239 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0725ed2be3e1cf2a5c229/lava-logs.yaml 2026-04-04 02:09:07,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:09:07,380 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:09:07,382 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0725ed2be3e1cf2a5c229/lava_callback.json.gz INFO: 172.20.0.1:58438 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:09:07,421 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:09:07,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0725ed2be3e1cf2a5c229 HTTP/1.1" 200 None 2026-04-04 02:09:07,437 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:09:07,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:09:07,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:09:07,482 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0725ed2be3e1cf2a5c229 HTTP/1.1" 200 None 2026-04-04 02:09:07,568 - lava_callback - INFO - Completed processing callback for node 69d0725ed2be3e1cf2a5c229 INFO: 172.20.0.1:59396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59400 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:09:17,009 - lava_callback - INFO - Received callback for node 69cdcfbf23eb053795d42c1a INFO: 172.20.0.1:59412 - "POST /node/69cdcfbf23eb053795d42c1a HTTP/1.0" 202 Accepted 2026-04-04 02:09:17,014 - lava_callback - INFO - Starting async job submit for node 69cdcfbf23eb053795d42c1a 2026-04-04 02:09:17,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:09:17,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcfbf23eb053795d42c1a HTTP/1.1" 200 None 2026-04-04 02:09:17,035 - lava_callback - DEBUG - Processing node 69cdcfbf23eb053795d42c1a: baseline-arm64-baylibre 2026-04-04 02:09:17,035 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdcfbf23eb053795d42c1a 2026-04-04 02:09:17,036 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:09:17,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdcfbf23eb053795d42c1a HTTP/1.1" 200 None 2026-04-04 02:09:17,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:09:17,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-04 02:09:17,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:09:17,107 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdcfbf23eb053795d42c1a HTTP/1.1" 200 None 2026-04-04 02:09:17,115 - lava_callback - ERROR - Error processing callback for node 69cdcfbf23eb053795d42c1a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60414 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:09:33,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:09:33,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:41270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43324 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:10:13,195 - lava_callback - INFO - Received callback for node 69d06eddd2be3e1cf2a5bf78 INFO: 172.20.0.1:54570 - "POST /node/69d06eddd2be3e1cf2a5bf78 HTTP/1.0" 202 Accepted 2026-04-04 02:10:13,202 - lava_callback - INFO - Starting async job submit for node 69d06eddd2be3e1cf2a5bf78 2026-04-04 02:10:13,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:10:13,283 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06eddd2be3e1cf2a5bf78 HTTP/1.1" 200 None 2026-04-04 02:10:13,284 - lava_callback - DEBUG - Processing node 69d06eddd2be3e1cf2a5bf78: baseline-arm 2026-04-04 02:10:13,963 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:10:14,091 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:10:14,093 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d06eddd2be3e1cf2a5bf78/log.txt.gz 2026-04-04 02:10:14,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:10:14,684 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:10:14,687 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d06eddd2be3e1cf2a5bf78/lava-logs.yaml 2026-04-04 02:10:14,702 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:10:14,871 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:10:14,873 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d06eddd2be3e1cf2a5bf78/lava_callback.json.gz DEBUG: Setup failed for 69d06eddd2be3e1cf2a5bf78. Transitting job node result: incomplete -> fail 2026-04-04 02:10:14,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:10:14,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d06eddd2be3e1cf2a5bf78 HTTP/1.1" 200 None 2026-04-04 02:10:14,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:10:15,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 02:10:15,014 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:10:15,039 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d06eddd2be3e1cf2a5bf78 HTTP/1.1" 200 None 2026-04-04 02:10:15,173 - lava_callback - INFO - Completed processing callback for node 69d06eddd2be3e1cf2a5bf78 INFO: 172.20.0.1:54572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51214 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:10:33,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:10:33,809 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:60604 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:10:37,754 - lava_callback - INFO - Received callback for node 69cdd01323eb053795d42c26 INFO: 172.20.0.1:60620 - "POST /node/69cdd01323eb053795d42c26 HTTP/1.0" 202 Accepted 2026-04-04 02:10:37,759 - lava_callback - INFO - Starting async job submit for node 69cdd01323eb053795d42c26 2026-04-04 02:10:37,760 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:10:37,777 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd01323eb053795d42c26 HTTP/1.1" 200 None 2026-04-04 02:10:37,778 - lava_callback - DEBUG - Processing node 69cdd01323eb053795d42c26: baseline-arm64-clabbe 2026-04-04 02:10:37,778 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd01323eb053795d42c26 2026-04-04 02:10:37,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:10:37,792 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd01323eb053795d42c26 HTTP/1.1" 200 None 2026-04-04 02:10:37,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:10:37,806 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-04 02:10:37,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:10:37,828 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd01323eb053795d42c26 HTTP/1.1" 200 None 2026-04-04 02:10:37,829 - lava_callback - ERROR - Error processing callback for node 69cdd01323eb053795d42c26: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32784 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:11:28,051 - lava_callback - INFO - Received callback for node 69d0718dd2be3e1cf2a5c1ae INFO: 172.20.0.1:32798 - "POST /node/69d0718dd2be3e1cf2a5c1ae HTTP/1.0" 202 Accepted 2026-04-04 02:11:28,059 - lava_callback - INFO - Starting async job submit for node 69d0718dd2be3e1cf2a5c1ae 2026-04-04 02:11:28,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:11:28,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0718dd2be3e1cf2a5c1ae HTTP/1.1" 200 None 2026-04-04 02:11:28,116 - lava_callback - DEBUG - Processing node 69d0718dd2be3e1cf2a5c1ae: baseline-arm64 2026-04-04 02:11:28,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:11:28,619 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:11:28,620 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0718dd2be3e1cf2a5c1ae/log.txt.gz 2026-04-04 02:11:28,841 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:11:29,079 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:11:29,082 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0718dd2be3e1cf2a5c1ae/lava-logs.yaml 2026-04-04 02:11:29,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:11:29,277 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:11:29,279 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d0718dd2be3e1cf2a5c1ae/lava_callback.json.gz DEBUG: Setup failed for 69d0718dd2be3e1cf2a5c1ae. Transitting job node result: incomplete -> fail 2026-04-04 02:11:29,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:11:29,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0718dd2be3e1cf2a5c1ae HTTP/1.1" 200 None 2026-04-04 02:11:29,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:11:29,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:11:29,413 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:11:29,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0718dd2be3e1cf2a5c1ae HTTP/1.1" 200 None 2026-04-04 02:11:29,513 - lava_callback - INFO - Completed processing callback for node 69d0718dd2be3e1cf2a5c1ae 2026-04-04 02:11:33,811 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:11:33,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:38268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33146 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:11:57,728 - lava_callback - INFO - Received callback for node 69cdd06023eb053795d42c2d INFO: 172.20.0.1:33154 - "POST /node/69cdd06023eb053795d42c2d HTTP/1.0" 202 Accepted 2026-04-04 02:11:57,733 - lava_callback - INFO - Starting async job submit for node 69cdd06023eb053795d42c2d 2026-04-04 02:11:57,734 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:11:57,753 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd06023eb053795d42c2d HTTP/1.1" 200 None 2026-04-04 02:11:57,754 - lava_callback - DEBUG - Processing node 69cdd06023eb053795d42c2d: baseline-arm64-clabbe 2026-04-04 02:11:57,754 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd06023eb053795d42c2d 2026-04-04 02:11:57,755 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:11:57,767 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd06023eb053795d42c2d HTTP/1.1" 200 None 2026-04-04 02:11:57,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:11:57,781 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-04 02:11:57,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:11:57,802 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd06023eb053795d42c2d HTTP/1.1" 200 None 2026-04-04 02:11:57,808 - lava_callback - ERROR - Error processing callback for node 69cdd06023eb053795d42c2d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42428 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54964 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:13:17,807 - lava_callback - INFO - Received callback for node 69cdd0ab23eb053795d42c38 INFO: 172.20.0.1:54966 - "POST /node/69cdd0ab23eb053795d42c38 HTTP/1.0" 202 Accepted 2026-04-04 02:13:17,816 - lava_callback - INFO - Starting async job submit for node 69cdd0ab23eb053795d42c38 2026-04-04 02:13:17,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:13:17,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd0ab23eb053795d42c38 HTTP/1.1" 200 None 2026-04-04 02:13:17,855 - lava_callback - DEBUG - Processing node 69cdd0ab23eb053795d42c38: baseline-arm64-clabbe 2026-04-04 02:13:17,855 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd0ab23eb053795d42c38 2026-04-04 02:13:17,856 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:13:17,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd0ab23eb053795d42c38 HTTP/1.1" 200 None 2026-04-04 02:13:17,878 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:13:17,898 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-04 02:13:17,900 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:13:17,932 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd0ab23eb053795d42c38 HTTP/1.1" 200 None 2026-04-04 02:13:17,935 - lava_callback - ERROR - Error processing callback for node 69cdd0ab23eb053795d42c38: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40776 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40784 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:14:17,842 - lava_callback - INFO - Received callback for node 69cdd0f723eb053795d42c4c INFO: 172.20.0.1:40800 - "POST /node/69cdd0f723eb053795d42c4c HTTP/1.0" 202 Accepted 2026-04-04 02:14:17,848 - lava_callback - INFO - Starting async job submit for node 69cdd0f723eb053795d42c4c 2026-04-04 02:14:17,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:14:17,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd0f723eb053795d42c4c HTTP/1.1" 200 None 2026-04-04 02:14:17,864 - lava_callback - DEBUG - Processing node 69cdd0f723eb053795d42c4c: baseline-arm64-clabbe 2026-04-04 02:14:17,864 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd0f723eb053795d42c4c 2026-04-04 02:14:17,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:14:17,878 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd0f723eb053795d42c4c HTTP/1.1" 200 None 2026-04-04 02:14:17,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:14:17,897 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-04 02:14:17,899 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:14:17,920 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd0f723eb053795d42c4c HTTP/1.1" 200 None 2026-04-04 02:14:17,921 - lava_callback - ERROR - Error processing callback for node 69cdd0f723eb053795d42c4c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43756 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:15:37,873 - lava_callback - INFO - Received callback for node 69cdd14523eb053795d42c5b INFO: 172.20.0.1:43772 - "POST /node/69cdd14523eb053795d42c5b HTTP/1.0" 202 Accepted 2026-04-04 02:15:37,879 - lava_callback - INFO - Starting async job submit for node 69cdd14523eb053795d42c5b 2026-04-04 02:15:37,880 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:15:37,894 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd14523eb053795d42c5b HTTP/1.1" 200 None 2026-04-04 02:15:37,895 - lava_callback - DEBUG - Processing node 69cdd14523eb053795d42c5b: baseline-arm64-clabbe 2026-04-04 02:15:37,895 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd14523eb053795d42c5b 2026-04-04 02:15:37,896 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:15:37,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd14523eb053795d42c5b HTTP/1.1" 200 None 2026-04-04 02:15:37,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:15:37,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-04 02:15:37,925 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:15:37,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd14523eb053795d42c5b HTTP/1.1" 200 None 2026-04-04 02:15:37,946 - lava_callback - ERROR - Error processing callback for node 69cdd14523eb053795d42c5b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35598 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:16:31,005 - lava_callback - INFO - Received callback for node 69d07219d2be3e1cf2a5c212 INFO: 172.20.0.1:39914 - "POST /node/69d07219d2be3e1cf2a5c212 HTTP/1.0" 202 Accepted 2026-04-04 02:16:31,011 - lava_callback - INFO - Starting async job submit for node 69d07219d2be3e1cf2a5c212 2026-04-04 02:16:31,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:16:31,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d07219d2be3e1cf2a5c212 HTTP/1.1" 200 None 2026-04-04 02:16:31,061 - lava_callback - DEBUG - Processing node 69d07219d2be3e1cf2a5c212: baseline-arm64 2026-04-04 02:16:31,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:16:31,817 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:16:31,819 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d07219d2be3e1cf2a5c212/log.txt.gz 2026-04-04 02:16:32,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:16:32,383 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:16:32,386 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d07219d2be3e1cf2a5c212/lava-logs.yaml 2026-04-04 02:16:32,399 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:16:32,601 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:16:32,603 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d07219d2be3e1cf2a5c212/lava_callback.json.gz 2026-04-04 02:16:32,640 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:16:32,657 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d07219d2be3e1cf2a5c212 HTTP/1.1" 200 None 2026-04-04 02:16:32,659 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:16:32,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:16:32,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:16:32,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d07219d2be3e1cf2a5c212 HTTP/1.1" 200 None 2026-04-04 02:16:32,766 - lava_callback - INFO - Completed processing callback for node 69d07219d2be3e1cf2a5c212 2026-04-04 02:16:33,837 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:16:33,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53396 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53406 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:18:17,941 - lava_callback - INFO - Received callback for node 69cdd1dd23eb053795d42c6c INFO: 172.20.0.1:53414 - "POST /node/69cdd1dd23eb053795d42c6c HTTP/1.0" 202 Accepted 2026-04-04 02:18:17,947 - lava_callback - INFO - Starting async job submit for node 69cdd1dd23eb053795d42c6c 2026-04-04 02:18:17,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:18:17,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd1dd23eb053795d42c6c HTTP/1.1" 200 None 2026-04-04 02:18:17,968 - lava_callback - DEBUG - Processing node 69cdd1dd23eb053795d42c6c: baseline-arm64-clabbe 2026-04-04 02:18:17,969 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd1dd23eb053795d42c6c 2026-04-04 02:18:17,969 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:18:17,983 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd1dd23eb053795d42c6c HTTP/1.1" 200 None 2026-04-04 02:18:17,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:18:17,997 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-04 02:18:17,999 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:18:18,024 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd1dd23eb053795d42c6c HTTP/1.1" 200 None 2026-04-04 02:18:18,025 - lava_callback - ERROR - Error processing callback for node 69cdd1dd23eb053795d42c6c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:19:43,385 - lava_callback - INFO - Received callback for node 69d072abd2be3e1cf2a5c267 INFO: 172.20.0.1:48124 - "POST /node/69d072abd2be3e1cf2a5c267 HTTP/1.0" 202 Accepted 2026-04-04 02:19:43,392 - lava_callback - INFO - Starting async job submit for node 69d072abd2be3e1cf2a5c267 2026-04-04 02:19:43,431 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:19:43,451 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d072abd2be3e1cf2a5c267 HTTP/1.1" 200 None 2026-04-04 02:19:43,452 - lava_callback - DEBUG - Processing node 69d072abd2be3e1cf2a5c267: baseline-nfs-arm64 2026-04-04 02:19:44,063 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:19:44,216 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:19:44,218 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d072abd2be3e1cf2a5c267/log.txt.gz 2026-04-04 02:19:44,553 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:19:44,813 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:19:44,815 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d072abd2be3e1cf2a5c267/lava-logs.yaml 2026-04-04 02:19:44,827 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:19:45,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:19:45,011 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d072abd2be3e1cf2a5c267/lava_callback.json.gz 2026-04-04 02:19:45,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:19:45,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d072abd2be3e1cf2a5c267 HTTP/1.1" 200 None 2026-04-04 02:19:45,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:19:45,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:19:45,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:19:45,099 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d072abd2be3e1cf2a5c267 HTTP/1.1" 200 None 2026-04-04 02:19:45,152 - lava_callback - INFO - Completed processing callback for node 69d072abd2be3e1cf2a5c267 INFO: 172.20.0.1:48130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38436 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:20:03,857 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:20:03,872 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35260 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54840 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:20:51,756 - lava_callback - INFO - Received callback for node 69d070f0d2be3e1cf2a5c10f INFO: 172.20.0.1:43946 - "POST /node/69d070f0d2be3e1cf2a5c10f HTTP/1.0" 202 Accepted 2026-04-04 02:20:51,805 - lava_callback - INFO - Starting async job submit for node 69d070f0d2be3e1cf2a5c10f 2026-04-04 02:20:51,829 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:20:51,843 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d070f0d2be3e1cf2a5c10f HTTP/1.1" 200 None 2026-04-04 02:20:51,844 - lava_callback - DEBUG - Processing node 69d070f0d2be3e1cf2a5c10f: baseline-arm-pengutronix 2026-04-04 02:20:51,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:20:52,126 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:20:52,128 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d070f0d2be3e1cf2a5c10f/log.txt.gz 2026-04-04 02:20:52,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:20:52,405 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:20:52,407 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d070f0d2be3e1cf2a5c10f/lava-logs.yaml 2026-04-04 02:20:52,411 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:20:52,560 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:20:52,562 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d070f0d2be3e1cf2a5c10f/lava_callback.json.gz DEBUG: Setup failed for 69d070f0d2be3e1cf2a5c10f. Transitting job node result: incomplete -> fail 2026-04-04 02:20:52,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:20:52,646 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d070f0d2be3e1cf2a5c10f HTTP/1.1" 200 None 2026-04-04 02:20:52,648 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:20:52,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 02:20:52,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:20:52,686 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d070f0d2be3e1cf2a5c10f HTTP/1.1" 200 None 2026-04-04 02:20:52,757 - lava_callback - INFO - Completed processing callback for node 69d070f0d2be3e1cf2a5c10f INFO: 172.20.0.1:43956 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:21:03,874 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:21:03,890 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:43714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43724 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:21:07,768 - lava_callback - INFO - Received callback for node 69d073e0d2be3e1cf2a5c344 INFO: 172.20.0.1:43736 - "POST /node/69d073e0d2be3e1cf2a5c344 HTTP/1.0" 202 Accepted 2026-04-04 02:21:07,775 - lava_callback - INFO - Starting async job submit for node 69d073e0d2be3e1cf2a5c344 2026-04-04 02:21:07,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:21:07,840 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d073e0d2be3e1cf2a5c344 HTTP/1.1" 200 None 2026-04-04 02:21:07,841 - lava_callback - DEBUG - Processing node 69d073e0d2be3e1cf2a5c344: baseline-arm64 2026-04-04 02:21:08,316 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:21:08,500 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:21:08,502 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d073e0d2be3e1cf2a5c344/log.txt.gz 2026-04-04 02:21:08,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:21:09,010 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:21:09,013 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d073e0d2be3e1cf2a5c344/lava-logs.yaml 2026-04-04 02:21:09,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:21:09,199 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:21:09,201 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d073e0d2be3e1cf2a5c344/lava_callback.json.gz DEBUG: Setup failed for 69d073e0d2be3e1cf2a5c344. Transitting job node result: incomplete -> fail 2026-04-04 02:21:09,281 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:21:09,295 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d073e0d2be3e1cf2a5c344 HTTP/1.1" 200 None 2026-04-04 02:21:09,297 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:21:09,310 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:21:09,311 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:21:09,334 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d073e0d2be3e1cf2a5c344 HTTP/1.1" 200 None 2026-04-04 02:21:09,405 - lava_callback - INFO - Completed processing callback for node 69d073e0d2be3e1cf2a5c344 INFO: 172.20.0.1:35796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47502 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:21:33,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:21:33,909 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:59686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59522 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45488 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40662 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48888 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48296 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40186 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41804 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58794 - "GET /node/698b65f90a16aea5a0dc7222%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44180 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:29:41,488 - lava_callback - INFO - Received callback for node 69d072eed2be3e1cf2a5c294 INFO: 172.20.0.1:41914 - "POST /node/69d072eed2be3e1cf2a5c294 HTTP/1.0" 202 Accepted 2026-04-04 02:29:41,501 - lava_callback - INFO - Starting async job submit for node 69d072eed2be3e1cf2a5c294 2026-04-04 02:29:41,562 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:29:41,577 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d072eed2be3e1cf2a5c294 HTTP/1.1" 200 None 2026-04-04 02:29:41,578 - lava_callback - DEBUG - Processing node 69d072eed2be3e1cf2a5c294: baseline-nfs-arm64 2026-04-04 02:29:42,814 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:29:43,019 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:29:43,021 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d072eed2be3e1cf2a5c294/log.txt.gz 2026-04-04 02:29:43,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:29:43,959 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:29:43,962 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d072eed2be3e1cf2a5c294/lava-logs.yaml 2026-04-04 02:29:43,984 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:29:44,191 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:29:44,193 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d072eed2be3e1cf2a5c294/lava_callback.json.gz DEBUG: Setup failed for 69d072eed2be3e1cf2a5c294. Transitting job node result: incomplete -> fail 2026-04-04 02:29:44,291 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:29:44,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d072eed2be3e1cf2a5c294 HTTP/1.1" 200 None 2026-04-04 02:29:44,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:29:44,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:29:44,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:29:44,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d072eed2be3e1cf2a5c294 HTTP/1.1" 200 None 2026-04-04 02:29:44,442 - lava_callback - INFO - Completed processing callback for node 69d072eed2be3e1cf2a5c294 INFO: 172.20.0.1:41920 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:29:51,551 - lava_callback - INFO - Received callback for node 69d07547d2be3e1cf2a5c414 INFO: 172.20.0.1:39414 - "POST /node/69d07547d2be3e1cf2a5c414 HTTP/1.0" 202 Accepted 2026-04-04 02:29:51,558 - lava_callback - INFO - Starting async job submit for node 69d07547d2be3e1cf2a5c414 2026-04-04 02:29:51,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:29:51,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d07547d2be3e1cf2a5c414 HTTP/1.1" 200 None 2026-04-04 02:29:51,597 - lava_callback - DEBUG - Processing node 69d07547d2be3e1cf2a5c414: baseline-arm64 2026-04-04 02:29:52,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:29:52,281 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:29:52,283 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm64-69d07547d2be3e1cf2a5c414/log.txt.gz 2026-04-04 02:29:52,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:29:52,856 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:29:52,858 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm64-69d07547d2be3e1cf2a5c414/lava-logs.yaml 2026-04-04 02:29:52,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:29:53,060 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:29:53,062 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm64-69d07547d2be3e1cf2a5c414/lava_callback.json.gz 2026-04-04 02:29:53,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:29:53,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d07547d2be3e1cf2a5c414 HTTP/1.1" 200 None 2026-04-04 02:29:53,124 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:29:53,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:29:53,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:29:53,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d07547d2be3e1cf2a5c414 HTTP/1.1" 200 None 2026-04-04 02:29:53,240 - lava_callback - INFO - Completed processing callback for node 69d07547d2be3e1cf2a5c414 INFO: 172.20.0.1:39424 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:30:03,913 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:30:03,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:46670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46676 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:30:11,325 - lava_callback - INFO - Received callback for node 69d07373d2be3e1cf2a5c2fe INFO: 172.20.0.1:42694 - "POST /node/69d07373d2be3e1cf2a5c2fe HTTP/1.0" 202 Accepted 2026-04-04 02:30:11,332 - lava_callback - INFO - Starting async job submit for node 69d07373d2be3e1cf2a5c2fe 2026-04-04 02:30:11,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:30:11,400 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d07373d2be3e1cf2a5c2fe HTTP/1.1" 200 None 2026-04-04 02:30:11,401 - lava_callback - DEBUG - Processing node 69d07373d2be3e1cf2a5c2fe: baseline-arm 2026-04-04 02:30:12,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:30:12,265 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:30:12,267 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-69d07373d2be3e1cf2a5c2fe/log.txt.gz 2026-04-04 02:30:12,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:30:12,948 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:30:12,950 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-69d07373d2be3e1cf2a5c2fe/lava-logs.yaml 2026-04-04 02:30:12,967 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:30:13,151 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:30:13,153 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-69d07373d2be3e1cf2a5c2fe/lava_callback.json.gz DEBUG: Setup failed for 69d07373d2be3e1cf2a5c2fe. Transitting job node result: incomplete -> fail 2026-04-04 02:30:13,273 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:30:13,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d07373d2be3e1cf2a5c2fe HTTP/1.1" 200 None 2026-04-04 02:30:13,287 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:30:13,301 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 02:30:13,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:30:13,327 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d07373d2be3e1cf2a5c2fe HTTP/1.1" 200 None 2026-04-04 02:30:13,422 - lava_callback - INFO - Completed processing callback for node 69d07373d2be3e1cf2a5c2fe INFO: 172.20.0.1:42708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54896 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:30:33,937 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:30:33,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:35866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37962 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51816 - "GET /node/6940c6881e3c24e95681d987%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51830 - "GET /node/698af48e3b3beb19e7616a44%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60978 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44974 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:32:47,842 - lava_callback - INFO - Received callback for node 69d075d3d2be3e1cf2a5c4a2 INFO: 172.20.0.1:44982 - "POST /node/69d075d3d2be3e1cf2a5c4a2 HTTP/1.0" 202 Accepted 2026-04-04 02:32:47,850 - lava_callback - INFO - Starting async job submit for node 69d075d3d2be3e1cf2a5c4a2 2026-04-04 02:32:47,890 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:32:47,905 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d075d3d2be3e1cf2a5c4a2 HTTP/1.1" 200 None 2026-04-04 02:32:47,906 - lava_callback - DEBUG - Processing node 69d075d3d2be3e1cf2a5c4a2: baseline-nfs-arm64 2026-04-04 02:32:48,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:32:48,696 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:32:48,698 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d075d3d2be3e1cf2a5c4a2/log.txt.gz 2026-04-04 02:32:48,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:32:49,187 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:32:49,189 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d075d3d2be3e1cf2a5c4a2/lava-logs.yaml 2026-04-04 02:32:49,201 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:32:49,371 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:32:49,373 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d075d3d2be3e1cf2a5c4a2/lava_callback.json.gz 2026-04-04 02:32:49,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:32:49,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d075d3d2be3e1cf2a5c4a2 HTTP/1.1" 200 None 2026-04-04 02:32:49,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:32:49,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:32:49,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:32:49,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d075d3d2be3e1cf2a5c4a2 HTTP/1.1" 200 None 2026-04-04 02:32:49,521 - lava_callback - INFO - Completed processing callback for node 69d075d3d2be3e1cf2a5c4a2 INFO: 172.20.0.1:34248 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:33:03,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:33:03,991 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:39624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39640 - "GET /node/689ed628df4198951e9e336e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:39644 - "GET /node/693afc37fee8383e92b6d844%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:56474 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53446 - "GET /node/693af710fee8383e92b6bea3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48558 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45556 - "GET /node/69b835f90b8ad4124cec39ea%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47404 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40770 - "GET /node/693b3e504b47c3f87266e3d3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42328 - "GET /node/693771e80c55536e7dddaaf8%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42338 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:41:38,732 - lava_callback - INFO - Received callback for node 69cdd75d23eb053795d42cf1 INFO: 172.20.0.1:42350 - "POST /node/69cdd75d23eb053795d42cf1 HTTP/1.0" 202 Accepted 2026-04-04 02:41:38,741 - lava_callback - INFO - Starting async job submit for node 69cdd75d23eb053795d42cf1 2026-04-04 02:41:38,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:41:38,757 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd75d23eb053795d42cf1 HTTP/1.1" 200 None 2026-04-04 02:41:38,758 - lava_callback - DEBUG - Processing node 69cdd75d23eb053795d42cf1: baseline-arm-baylibre 2026-04-04 02:41:38,759 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd75d23eb053795d42cf1 2026-04-04 02:41:38,759 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:41:38,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd75d23eb053795d42cf1 HTTP/1.1" 200 None 2026-04-04 02:41:38,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:41:38,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-04 02:41:38,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:41:38,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd75d23eb053795d42cf1 HTTP/1.1" 200 None 2026-04-04 02:41:38,832 - lava_callback - ERROR - Error processing callback for node 69cdd75d23eb053795d42cf1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36742 - "GET /node/698de563594ec372d06985fd%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56436 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56452 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:42:18,612 - lava_callback - INFO - Received callback for node 69cdd78523eb053795d42d00 INFO: 172.20.0.1:56458 - "POST /node/69cdd78523eb053795d42d00 HTTP/1.0" 202 Accepted 2026-04-04 02:42:18,618 - lava_callback - INFO - Starting async job submit for node 69cdd78523eb053795d42d00 2026-04-04 02:42:18,619 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:42:18,633 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd78523eb053795d42d00 HTTP/1.1" 200 None 2026-04-04 02:42:18,634 - lava_callback - DEBUG - Processing node 69cdd78523eb053795d42d00: baseline-arm-baylibre 2026-04-04 02:42:18,634 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd78523eb053795d42d00 2026-04-04 02:42:18,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:42:18,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd78523eb053795d42d00 HTTP/1.1" 200 None 2026-04-04 02:42:18,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:42:18,666 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-04 02:42:18,669 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:42:18,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd78523eb053795d42d00 HTTP/1.1" 200 None 2026-04-04 02:42:18,698 - lava_callback - ERROR - Error processing callback for node 69cdd78523eb053795d42d00: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59596 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:42:52,675 - lava_callback - INFO - Received callback for node 69d0761cd2be3e1cf2a5c4bc INFO: 172.20.0.1:50916 - "POST /node/69d0761cd2be3e1cf2a5c4bc HTTP/1.0" 202 Accepted 2026-04-04 02:42:52,721 - lava_callback - INFO - Starting async job submit for node 69d0761cd2be3e1cf2a5c4bc 2026-04-04 02:42:52,745 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:42:52,760 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0761cd2be3e1cf2a5c4bc HTTP/1.1" 200 None 2026-04-04 02:42:52,761 - lava_callback - DEBUG - Processing node 69d0761cd2be3e1cf2a5c4bc: baseline-arm-pengutronix 2026-04-04 02:42:52,865 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:42:52,995 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:42:52,997 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d0761cd2be3e1cf2a5c4bc/log.txt.gz 2026-04-04 02:42:53,073 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 INFO: 172.20.0.1:50928 - "GET /node/693b0f13fee8383e92b72a2e%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50930 - "GET /node/69c02895c3d73288b0d08116%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50932 - "GET / HTTP/1.0" 200 OK Upload attempt 1 failed with ConnectionError: HTTPSConnectionPool(host='files-staging.kernelci.org', port=443): Max retries exceeded with url: /upload (Caused by NameResolutionError(": Failed to resolve 'files-staging.kernelci.org' ([Errno -2] Name or service not known)")) Retrying in 10 seconds... (4 retries remaining) INFO: 172.20.0.1:55312 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:43:07,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:43:07,233 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:43:07,233 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d0761cd2be3e1cf2a5c4bc/lava-logs.yaml 2026-04-04 02:43:07,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:43:07,381 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:43:07,383 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-arm-pengutronix-69d0761cd2be3e1cf2a5c4bc/lava_callback.json.gz DEBUG: Setup failed for 69d0761cd2be3e1cf2a5c4bc. Transitting job node result: incomplete -> fail 2026-04-04 02:43:07,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:43:07,446 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d0761cd2be3e1cf2a5c4bc HTTP/1.1" 200 None 2026-04-04 02:43:07,448 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:43:07,461 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058cbd2be3e1cf2a5a7ef HTTP/1.1" 200 None 2026-04-04 02:43:07,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:43:07,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d0761cd2be3e1cf2a5c4bc HTTP/1.1" 200 None 2026-04-04 02:43:07,545 - lava_callback - INFO - Completed processing callback for node 69d0761cd2be3e1cf2a5c4bc INFO: 172.20.0.1:55328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43768 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36966 - "GET /node/69876b96e31b58f834f19822%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:36978 - "GET /node/693af588fee8383e92b6b3f1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:36980 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:43:33,995 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:43:34,012 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:55378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47692 - "GET /node/6982ace63507dcddca5906ca%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47700 - "GET /node/693710b093bfb52307cd3ae4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47708 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45594 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:45:39,286 - lava_callback - INFO - Received callback for node 69cdd83f23eb053795d42d39 INFO: 172.20.0.1:45608 - "POST /node/69cdd83f23eb053795d42d39 HTTP/1.0" 202 Accepted 2026-04-04 02:45:39,291 - lava_callback - INFO - Starting async job submit for node 69cdd83f23eb053795d42d39 2026-04-04 02:45:39,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:45:39,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd83f23eb053795d42d39 HTTP/1.1" 200 None 2026-04-04 02:45:39,307 - lava_callback - DEBUG - Processing node 69cdd83f23eb053795d42d39: baseline-arm-clabbe 2026-04-04 02:45:39,308 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd83f23eb053795d42d39 2026-04-04 02:45:39,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:45:39,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd83f23eb053795d42d39 HTTP/1.1" 200 None 2026-04-04 02:45:39,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:45:39,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-04 02:45:39,337 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:45:39,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd83f23eb053795d42d39 HTTP/1.1" 200 None 2026-04-04 02:45:39,367 - lava_callback - ERROR - Error processing callback for node 69cdd83f23eb053795d42d39: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40212 - "GET /node/6929a22709677735f68afa4c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40220 - "GET /node/69a3fd276ebb1cc56c1def4b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59872 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:45:59,174 - lava_callback - INFO - Received callback for node 69cdd86323eb053795d42d47 INFO: 172.20.0.1:59874 - "POST /node/69cdd86323eb053795d42d47 HTTP/1.0" 202 Accepted 2026-04-04 02:45:59,179 - lava_callback - INFO - Starting async job submit for node 69cdd86323eb053795d42d47 2026-04-04 02:45:59,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:45:59,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd86323eb053795d42d47 HTTP/1.1" 200 None 2026-04-04 02:45:59,201 - lava_callback - DEBUG - Processing node 69cdd86323eb053795d42d47: baseline-arm-clabbe 2026-04-04 02:45:59,202 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd86323eb053795d42d47 2026-04-04 02:45:59,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:45:59,216 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd86323eb053795d42d47 HTTP/1.1" 200 None 2026-04-04 02:45:59,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:45:59,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-04 02:45:59,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:45:59,254 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd86323eb053795d42d47 HTTP/1.1" 200 None 2026-04-04 02:45:59,259 - lava_callback - ERROR - Error processing callback for node 69cdd86323eb053795d42d47: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58688 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:46:39,214 - lava_callback - INFO - Received callback for node 69cdd88923eb053795d42d52 INFO: 172.20.0.1:58692 - "POST /node/69cdd88923eb053795d42d52 HTTP/1.0" 202 Accepted 2026-04-04 02:46:39,220 - lava_callback - INFO - Starting async job submit for node 69cdd88923eb053795d42d52 2026-04-04 02:46:39,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:46:39,235 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd88923eb053795d42d52 HTTP/1.1" 200 None 2026-04-04 02:46:39,236 - lava_callback - DEBUG - Processing node 69cdd88923eb053795d42d52: baseline-arm-clabbe 2026-04-04 02:46:39,236 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd88923eb053795d42d52 2026-04-04 02:46:39,237 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:46:39,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd88923eb053795d42d52 HTTP/1.1" 200 None 2026-04-04 02:46:39,256 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:46:39,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-04 02:46:39,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:46:39,302 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd88923eb053795d42d52 HTTP/1.1" 200 None 2026-04-04 02:46:39,304 - lava_callback - ERROR - Error processing callback for node 69cdd88923eb053795d42d52: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57114 - "GET /node/693ae3a2fee8383e92b641ad%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57116 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57130 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:47:19,264 - lava_callback - INFO - Received callback for node 69cdd8b023eb053795d42d5d INFO: 172.20.0.1:57140 - "POST /node/69cdd8b023eb053795d42d5d HTTP/1.0" 202 Accepted 2026-04-04 02:47:19,270 - lava_callback - INFO - Starting async job submit for node 69cdd8b023eb053795d42d5d 2026-04-04 02:47:19,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:47:19,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd8b023eb053795d42d5d HTTP/1.1" 200 None 2026-04-04 02:47:19,298 - lava_callback - DEBUG - Processing node 69cdd8b023eb053795d42d5d: baseline-arm-clabbe 2026-04-04 02:47:19,298 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd8b023eb053795d42d5d 2026-04-04 02:47:19,299 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:47:19,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd8b023eb053795d42d5d HTTP/1.1" 200 None 2026-04-04 02:47:19,313 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:47:19,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-04 02:47:19,328 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:47:19,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd8b023eb053795d42d5d HTTP/1.1" 200 None 2026-04-04 02:47:19,354 - lava_callback - ERROR - Error processing callback for node 69cdd8b023eb053795d42d5d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47742 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:47:59,417 - lava_callback - INFO - Received callback for node 69cdd8d423eb053795d42d66 INFO: 172.20.0.1:47756 - "POST /node/69cdd8d423eb053795d42d66 HTTP/1.0" 202 Accepted 2026-04-04 02:47:59,423 - lava_callback - INFO - Starting async job submit for node 69cdd8d423eb053795d42d66 2026-04-04 02:47:59,424 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:47:59,438 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd8d423eb053795d42d66 HTTP/1.1" 200 None 2026-04-04 02:47:59,439 - lava_callback - DEBUG - Processing node 69cdd8d423eb053795d42d66: baseline-arm-clabbe 2026-04-04 02:47:59,439 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd8d423eb053795d42d66 2026-04-04 02:47:59,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:47:59,453 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd8d423eb053795d42d66 HTTP/1.1" 200 None 2026-04-04 02:47:59,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:47:59,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-04 02:47:59,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:47:59,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd8d423eb053795d42d66 HTTP/1.1" 200 None 2026-04-04 02:47:59,495 - lava_callback - ERROR - Error processing callback for node 69cdd8d423eb053795d42d66: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35778 - "GET /node/693af5d7fee8383e92b6b58c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:35792 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:48:39,307 - lava_callback - INFO - Received callback for node 69cdd8f823eb053795d42d71 INFO: 172.20.0.1:35806 - "POST /node/69cdd8f823eb053795d42d71 HTTP/1.0" 202 Accepted 2026-04-04 02:48:39,316 - lava_callback - INFO - Starting async job submit for node 69cdd8f823eb053795d42d71 2026-04-04 02:48:39,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:48:39,337 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd8f823eb053795d42d71 HTTP/1.1" 200 None 2026-04-04 02:48:39,338 - lava_callback - DEBUG - Processing node 69cdd8f823eb053795d42d71: baseline-arm-clabbe 2026-04-04 02:48:39,339 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdd8f823eb053795d42d71 2026-04-04 02:48:39,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:48:39,353 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdd8f823eb053795d42d71 HTTP/1.1" 200 None 2026-04-04 02:48:39,355 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:48:39,373 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd48274bda3ec93a41266 HTTP/1.1" 200 None 2026-04-04 02:48:39,375 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:48:39,403 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdd8f823eb053795d42d71 HTTP/1.1" 200 None 2026-04-04 02:48:39,404 - lava_callback - ERROR - Error processing callback for node 69cdd8f823eb053795d42d71: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37632 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42982 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:49:40,382 - lava_callback - INFO - Received callback for node 69d077a1d2be3e1cf2a5c4db INFO: 172.20.0.1:59982 - "POST /node/69d077a1d2be3e1cf2a5c4db HTTP/1.0" 202 Accepted 2026-04-04 02:49:40,390 - lava_callback - INFO - Starting async job submit for node 69d077a1d2be3e1cf2a5c4db 2026-04-04 02:49:40,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:49:40,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d077a1d2be3e1cf2a5c4db HTTP/1.1" 200 None 2026-04-04 02:49:40,455 - lava_callback - DEBUG - Processing node 69d077a1d2be3e1cf2a5c4db: baseline-nfs-arm64 2026-04-04 02:49:41,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:49:41,877 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:49:41,879 - lava_callback - INFO - Log uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d077a1d2be3e1cf2a5c4db/log.txt.gz 2026-04-04 02:49:42,527 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:49:42,753 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:49:42,756 - lava_callback - INFO - LAVA YAML uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d077a1d2be3e1cf2a5c4db/lava-logs.yaml 2026-04-04 02:49:42,780 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): files-staging.kernelci.org:443 2026-04-04 02:49:43,095 - urllib3.connectionpool - DEBUG - https://files-staging.kernelci.org:443 "POST /upload HTTP/1.1" 200 0 2026-04-04 02:49:43,097 - lava_callback - INFO - Callback data uploaded to https://files-staging.kernelci.org/baseline-nfs-arm64-69d077a1d2be3e1cf2a5c4db/lava_callback.json.gz DEBUG: Setup failed for 69d077a1d2be3e1cf2a5c4db. Transitting job node result: incomplete -> fail 2026-04-04 02:49:43,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:49:43,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d077a1d2be3e1cf2a5c4db HTTP/1.1" 200 None 2026-04-04 02:49:43,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:49:43,249 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69d058d7d2be3e1cf2a5a804 HTTP/1.1" 200 None 2026-04-04 02:49:43,251 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:49:43,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69d077a1d2be3e1cf2a5c4db HTTP/1.1" 200 None 2026-04-04 02:49:43,386 - lava_callback - INFO - Completed processing callback for node 69d077a1d2be3e1cf2a5c4db INFO: 172.20.0.1:59996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35738 - "GET / HTTP/1.0" 200 OK 2026-04-04 02:50:04,015 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 02:50:04,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "POST /latest/telemetry HTTP/1.1" 200 14 INFO: 172.20.0.1:45158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36566 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58626 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50828 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38894 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40024 - "GET /node/688eafc23a66ab4492e8e9db%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34800 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38540 - "GET /node/69831247409235c29c0de89b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:38556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35552 - "GET /node/69876fb8e31b58f834f1a629%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56242 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46432 - "GET /node/693b19cb15f407b7629df7b4%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:46442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48016 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40682 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59952 - "GET /node/6988c0839fc35f969e1ed879%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:59956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33270 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49398 - "GET /node/698af2f13b3beb19e7616887%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:59334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34118 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55586 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38652 - "GET /node/69933e3131166481c8dedc65%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40592 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54060 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52060 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58080 - "GET / HTTP/1.0" 200 OK 2026-04-04 03:18:00,655 - lava_callback - INFO - Received callback for node 69cddfdb23eb053795d42df5 INFO: 172.20.0.1:34394 - "POST /node/69cddfdb23eb053795d42df5 HTTP/1.0" 202 Accepted 2026-04-04 03:18:00,661 - lava_callback - INFO - Starting async job submit for node 69cddfdb23eb053795d42df5 2026-04-04 03:18:00,662 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:18:00,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cddfdb23eb053795d42df5 HTTP/1.1" 200 None 2026-04-04 03:18:00,678 - lava_callback - DEBUG - Processing node 69cddfdb23eb053795d42df5: baseline-arm64-baylibre 2026-04-04 03:18:00,678 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cddfdb23eb053795d42df5 2026-04-04 03:18:00,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:18:00,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cddfdb23eb053795d42df5 HTTP/1.1" 200 None 2026-04-04 03:18:00,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:18:00,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-04 03:18:00,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:18:04,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cddfdb23eb053795d42df5 HTTP/1.1" 200 None 2026-04-04 03:18:04,745 - lava_callback - ERROR - Error processing callback for node 69cddfdb23eb053795d42df5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43180 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52150 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44664 - "GET / HTTP/1.0" 200 OK 2026-04-04 03:21:00,689 - lava_callback - INFO - Received callback for node 69cde09023eb053795d42e04 INFO: 172.20.0.1:58584 - "POST /node/69cde09023eb053795d42e04 HTTP/1.0" 202 Accepted 2026-04-04 03:21:00,694 - lava_callback - INFO - Starting async job submit for node 69cde09023eb053795d42e04 2026-04-04 03:21:00,695 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:21:00,712 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde09023eb053795d42e04 HTTP/1.1" 200 None 2026-04-04 03:21:00,713 - lava_callback - DEBUG - Processing node 69cde09023eb053795d42e04: baseline-arm64-baylibre 2026-04-04 03:21:00,713 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cde09023eb053795d42e04 2026-04-04 03:21:00,714 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:21:00,726 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde09023eb053795d42e04 HTTP/1.1" 200 None 2026-04-04 03:21:00,728 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:21:00,740 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-04 03:21:00,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:21:00,766 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cde09023eb053795d42e04 HTTP/1.1" 200 None 2026-04-04 03:21:00,773 - lava_callback - ERROR - Error processing callback for node 69cde09023eb053795d42e04: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58560 - "GET / HTTP/1.0" 200 OK 2026-04-04 03:22:20,662 - lava_callback - INFO - Received callback for node 69cde0e423eb053795d42e0f INFO: 172.20.0.1:47988 - "POST /node/69cde0e423eb053795d42e0f HTTP/1.0" 202 Accepted 2026-04-04 03:22:20,671 - lava_callback - INFO - Starting async job submit for node 69cde0e423eb053795d42e0f 2026-04-04 03:22:20,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:22:20,702 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde0e423eb053795d42e0f HTTP/1.1" 200 None 2026-04-04 03:22:20,703 - lava_callback - DEBUG - Processing node 69cde0e423eb053795d42e0f: baseline-arm64-baylibre 2026-04-04 03:22:20,704 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cde0e423eb053795d42e0f 2026-04-04 03:22:20,704 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:22:20,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde0e423eb053795d42e0f HTTP/1.1" 200 None 2026-04-04 03:22:20,731 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:22:20,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-04 03:22:20,747 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:22:20,775 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cde0e423eb053795d42e0f HTTP/1.1" 200 None 2026-04-04 03:22:20,777 - lava_callback - ERROR - Error processing callback for node 69cde0e423eb053795d42e0f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54862 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57916 - "GET / HTTP/1.0" 200 OK 2026-04-04 03:24:00,741 - lava_callback - INFO - Received callback for node 69cde14623eb053795d42e1b INFO: 172.20.0.1:33244 - "POST /node/69cde14623eb053795d42e1b HTTP/1.0" 202 Accepted 2026-04-04 03:24:00,749 - lava_callback - INFO - Starting async job submit for node 69cde14623eb053795d42e1b 2026-04-04 03:24:00,750 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:24:00,763 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde14623eb053795d42e1b HTTP/1.1" 200 None 2026-04-04 03:24:00,764 - lava_callback - DEBUG - Processing node 69cde14623eb053795d42e1b: baseline-arm64-baylibre 2026-04-04 03:24:00,764 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cde14623eb053795d42e1b 2026-04-04 03:24:00,765 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:24:00,778 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde14623eb053795d42e1b HTTP/1.1" 200 None 2026-04-04 03:24:00,779 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:24:00,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-04 03:24:00,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:24:00,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cde14623eb053795d42e1b HTTP/1.1" 200 None 2026-04-04 03:24:00,821 - lava_callback - ERROR - Error processing callback for node 69cde14623eb053795d42e1b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41766 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48360 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34320 - "GET / HTTP/1.0" 200 OK 2026-04-04 03:26:00,929 - lava_callback - INFO - Received callback for node 69cde1bf23eb053795d42e2c INFO: 172.20.0.1:34216 - "POST /node/69cde1bf23eb053795d42e2c HTTP/1.0" 202 Accepted 2026-04-04 03:26:00,935 - lava_callback - INFO - Starting async job submit for node 69cde1bf23eb053795d42e2c 2026-04-04 03:26:00,936 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:26:00,951 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde1bf23eb053795d42e2c HTTP/1.1" 200 None 2026-04-04 03:26:00,951 - lava_callback - DEBUG - Processing node 69cde1bf23eb053795d42e2c: baseline-arm64-baylibre 2026-04-04 03:26:00,952 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cde1bf23eb053795d42e2c 2026-04-04 03:26:00,952 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:26:00,967 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde1bf23eb053795d42e2c HTTP/1.1" 200 None 2026-04-04 03:26:00,968 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:26:00,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-04 03:26:00,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:26:01,005 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cde1bf23eb053795d42e2c HTTP/1.1" 200 None 2026-04-04 03:26:01,006 - lava_callback - ERROR - Error processing callback for node 69cde1bf23eb053795d42e2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49564 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48018 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53810 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48618 - "GET / HTTP/1.0" 200 OK 2026-04-04 03:29:01,166 - lava_callback - INFO - Received callback for node 69cde27923eb053795d42e3b INFO: 172.20.0.1:33360 - "POST /node/69cde27923eb053795d42e3b HTTP/1.0" 202 Accepted 2026-04-04 03:29:01,172 - lava_callback - INFO - Starting async job submit for node 69cde27923eb053795d42e3b 2026-04-04 03:29:01,173 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:29:01,192 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde27923eb053795d42e3b HTTP/1.1" 200 None 2026-04-04 03:29:01,193 - lava_callback - DEBUG - Processing node 69cde27923eb053795d42e3b: baseline-arm64-clabbe 2026-04-04 03:29:01,193 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cde27923eb053795d42e3b 2026-04-04 03:29:01,194 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:29:01,207 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde27923eb053795d42e3b HTTP/1.1" 200 None 2026-04-04 03:29:01,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:29:01,226 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-04 03:29:01,227 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:29:01,250 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cde27923eb053795d42e3b HTTP/1.1" 200 None 2026-04-04 03:29:01,252 - lava_callback - ERROR - Error processing callback for node 69cde27923eb053795d42e3b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37440 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56994 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46742 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45304 - "GET / HTTP/1.0" 200 OK 2026-04-04 03:32:01,070 - lava_callback - INFO - Received callback for node 69cde32e23eb053795d42e49 INFO: 172.20.0.1:40856 - "POST /node/69cde32e23eb053795d42e49 HTTP/1.0" 202 Accepted 2026-04-04 03:32:01,076 - lava_callback - INFO - Starting async job submit for node 69cde32e23eb053795d42e49 2026-04-04 03:32:01,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:32:01,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde32e23eb053795d42e49 HTTP/1.1" 200 None 2026-04-04 03:32:01,092 - lava_callback - DEBUG - Processing node 69cde32e23eb053795d42e49: baseline-arm64-clabbe 2026-04-04 03:32:01,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cde32e23eb053795d42e49 2026-04-04 03:32:01,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:32:01,105 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde32e23eb053795d42e49 HTTP/1.1" 200 None 2026-04-04 03:32:01,106 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:32:01,119 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-04 03:32:01,121 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:32:01,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cde32e23eb053795d42e49 HTTP/1.1" 200 None 2026-04-04 03:32:01,163 - lava_callback - ERROR - Error processing callback for node 69cde32e23eb053795d42e49: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36358 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42324 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42334 - "GET / HTTP/1.0" 200 OK 2026-04-04 03:33:21,129 - lava_callback - INFO - Received callback for node 69cde37b23eb053795d42e51 INFO: 172.20.0.1:40826 - "POST /node/69cde37b23eb053795d42e51 HTTP/1.0" 202 Accepted 2026-04-04 03:33:21,134 - lava_callback - INFO - Starting async job submit for node 69cde37b23eb053795d42e51 2026-04-04 03:33:21,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:33:21,150 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde37b23eb053795d42e51 HTTP/1.1" 200 None 2026-04-04 03:33:21,151 - lava_callback - DEBUG - Processing node 69cde37b23eb053795d42e51: baseline-arm64-clabbe 2026-04-04 03:33:21,151 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cde37b23eb053795d42e51 2026-04-04 03:33:21,152 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:33:21,173 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde37b23eb053795d42e51 HTTP/1.1" 200 None 2026-04-04 03:33:21,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:33:21,187 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-04 03:33:21,188 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:33:21,208 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cde37b23eb053795d42e51 HTTP/1.1" 200 None 2026-04-04 03:33:21,214 - lava_callback - ERROR - Error processing callback for node 69cde37b23eb053795d42e51: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47834 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60256 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48660 - "GET / HTTP/1.0" 200 OK 2026-04-04 03:36:01,192 - lava_callback - INFO - Received callback for node 69cde41e23eb053795d42e61 INFO: 172.20.0.1:49060 - "POST /node/69cde41e23eb053795d42e61 HTTP/1.0" 202 Accepted 2026-04-04 03:36:01,197 - lava_callback - INFO - Starting async job submit for node 69cde41e23eb053795d42e61 2026-04-04 03:36:01,198 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:36:01,213 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde41e23eb053795d42e61 HTTP/1.1" 200 None 2026-04-04 03:36:01,214 - lava_callback - DEBUG - Processing node 69cde41e23eb053795d42e61: baseline-arm64-clabbe 2026-04-04 03:36:01,214 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cde41e23eb053795d42e61 2026-04-04 03:36:01,214 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:36:01,227 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde41e23eb053795d42e61 HTTP/1.1" 200 None 2026-04-04 03:36:01,229 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:36:01,242 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-04 03:36:01,243 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:36:01,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cde41e23eb053795d42e61 HTTP/1.1" 200 None 2026-04-04 03:36:01,265 - lava_callback - ERROR - Error processing callback for node 69cde41e23eb053795d42e61: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36736 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54390 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54396 - "GET / HTTP/1.0" 200 OK 2026-04-04 03:37:21,249 - lava_callback - INFO - Received callback for node 69cde46a23eb053795d42e69 INFO: 172.20.0.1:59476 - "POST /node/69cde46a23eb053795d42e69 HTTP/1.0" 202 Accepted 2026-04-04 03:37:21,254 - lava_callback - INFO - Starting async job submit for node 69cde46a23eb053795d42e69 2026-04-04 03:37:21,255 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:37:21,269 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde46a23eb053795d42e69 HTTP/1.1" 200 None 2026-04-04 03:37:21,270 - lava_callback - DEBUG - Processing node 69cde46a23eb053795d42e69: baseline-arm64-clabbe 2026-04-04 03:37:21,270 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cde46a23eb053795d42e69 2026-04-04 03:37:21,271 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:37:21,290 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde46a23eb053795d42e69 HTTP/1.1" 200 None 2026-04-04 03:37:21,292 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:37:21,304 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-04 03:37:21,306 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:37:21,326 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cde46a23eb053795d42e69 HTTP/1.1" 200 None 2026-04-04 03:37:21,327 - lava_callback - ERROR - Error processing callback for node 69cde46a23eb053795d42e69: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42486 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55684 - "GET / HTTP/1.0" 200 OK 2026-04-04 03:39:01,333 - lava_callback - INFO - Received callback for node 69cde4d123eb053795d42e79 INFO: 172.20.0.1:42096 - "POST /node/69cde4d123eb053795d42e79 HTTP/1.0" 202 Accepted 2026-04-04 03:39:01,339 - lava_callback - INFO - Starting async job submit for node 69cde4d123eb053795d42e79 2026-04-04 03:39:01,340 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:39:01,354 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde4d123eb053795d42e79 HTTP/1.1" 200 None 2026-04-04 03:39:01,355 - lava_callback - DEBUG - Processing node 69cde4d123eb053795d42e79: baseline-arm64-clabbe 2026-04-04 03:39:01,355 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cde4d123eb053795d42e79 2026-04-04 03:39:01,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:39:01,369 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cde4d123eb053795d42e79 HTTP/1.1" 200 None 2026-04-04 03:39:01,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:39:01,393 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ccd49074bda3ec93a41275 HTTP/1.1" 200 None 2026-04-04 03:39:01,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 03:39:01,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cde4d123eb053795d42e79 HTTP/1.1" 200 None 2026-04-04 03:39:01,416 - lava_callback - ERROR - Error processing callback for node 69cde4d123eb053795d42e79: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36228 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40496 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33462 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33474 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44728 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58058 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52226 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52230 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36436 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40492 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55254 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33796 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53986 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55670 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43512 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58648 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36138 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56210 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48580 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58234 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52400 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34590 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34080 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53310 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53452 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34866 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38296 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53508 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60048 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44864 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47836 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36120 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49960 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34816 - "GET /node/69493eedebaed71d4380827a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57232 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56816 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57284 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54732 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57622 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54668 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38258 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41064 - "GET /node/694dda088177197a294f0401%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43948 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53672 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49954 - "GET /node/69113f17f15111fc8ce4796a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:49956 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43982 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36598 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59242 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51004 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51018 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47620 - "GET /node/69930bcc31166481c8de2715%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:47632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38290 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46148 - "GET /node/6928c37f3a49357e496c8154%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:46154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50842 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43894 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33908 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52252 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56306 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39600 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50950 - "GET /node/698f834d3013abef95d015e0%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:40354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47598 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57752 - "GET /node/693b0392fee8383e92b6f4d6%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39878 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52082 - "GET /node/69c025dac3d73288b0d07f8f%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:52088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34840 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34736 - "GET /node/69afdb75ae557f2600f0f77a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:34750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38630 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55968 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46190 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55132 - "GET /node/698b68200a16aea5a0dc7667%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55144 - "GET /node/693f032624745582e3995d07%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:55152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54710 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54720 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49338 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49352 - "GET /node/69281072b1b5e06bf6c04f43%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:50944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45688 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41706 - "GET /node/694b03ef6c96ed51d3173172%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55868 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34792 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53264 - "GET /node/6993c5613e9ba4930770e4d2%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53266 - "GET /node/69876c8ce31b58f834f19984%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51202 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49804 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43954 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43970 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43974 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54354 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58300 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34816 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34824 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49788 - "GET /node/6985d08671989fb141499b89%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:49798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42144 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42168 - "GET /node/6992688c98a8bdb31c39222b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:37320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50164 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44098 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38212 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46506 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38502 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38514 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42098 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40408 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40926 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55618 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35532 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33968 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54998 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:02:06,264 - lava_callback - INFO - Received callback for node 69cdf83a23eb053795d42edc INFO: 172.20.0.1:42158 - "POST /node/69cdf83a23eb053795d42edc HTTP/1.0" 202 Accepted 2026-04-04 05:02:06,269 - lava_callback - INFO - Starting async job submit for node 69cdf83a23eb053795d42edc 2026-04-04 05:02:06,270 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:02:06,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf83a23eb053795d42edc HTTP/1.1" 200 None 2026-04-04 05:02:06,285 - lava_callback - DEBUG - Processing node 69cdf83a23eb053795d42edc: baseline-arm-baylibre 2026-04-04 05:02:06,285 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf83a23eb053795d42edc 2026-04-04 05:02:06,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:02:06,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf83a23eb053795d42edc HTTP/1.1" 200 None 2026-04-04 05:02:06,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:02:06,313 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:02:06,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:02:06,336 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf83a23eb053795d42edc HTTP/1.1" 200 None 2026-04-04 05:02:06,337 - lava_callback - ERROR - Error processing callback for node 69cdf83a23eb053795d42edc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58066 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45666 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51658 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:03:06,224 - lava_callback - INFO - Received callback for node 69cdf87623eb053795d42ee8 INFO: 172.20.0.1:51172 - "POST /node/69cdf87623eb053795d42ee8 HTTP/1.0" 202 Accepted 2026-04-04 05:03:06,230 - lava_callback - INFO - Starting async job submit for node 69cdf87623eb053795d42ee8 2026-04-04 05:03:06,231 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:03:06,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf87623eb053795d42ee8 HTTP/1.1" 200 None 2026-04-04 05:03:06,246 - lava_callback - DEBUG - Processing node 69cdf87623eb053795d42ee8: baseline-arm-baylibre 2026-04-04 05:03:06,247 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf87623eb053795d42ee8 2026-04-04 05:03:06,247 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:03:06,260 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf87623eb053795d42ee8 HTTP/1.1" 200 None 2026-04-04 05:03:06,262 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:03:06,285 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:03:06,286 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:03:06,306 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf87623eb053795d42ee8 HTTP/1.1" 200 None 2026-04-04 05:03:06,310 - lava_callback - ERROR - Error processing callback for node 69cdf87623eb053795d42ee8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36024 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49112 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:04:06,559 - lava_callback - INFO - Received callback for node 69cdf8b123eb053795d42ef6 INFO: 172.20.0.1:49122 - "POST /node/69cdf8b123eb053795d42ef6 HTTP/1.0" 202 Accepted 2026-04-04 05:04:06,565 - lava_callback - INFO - Starting async job submit for node 69cdf8b123eb053795d42ef6 2026-04-04 05:04:06,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:04:06,581 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf8b123eb053795d42ef6 HTTP/1.1" 200 None 2026-04-04 05:04:06,582 - lava_callback - DEBUG - Processing node 69cdf8b123eb053795d42ef6: baseline-arm-baylibre 2026-04-04 05:04:06,582 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf8b123eb053795d42ef6 2026-04-04 05:04:06,583 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:04:06,596 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf8b123eb053795d42ef6 HTTP/1.1" 200 None 2026-04-04 05:04:06,598 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:04:06,612 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:04:06,613 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:04:06,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf8b123eb053795d42ef6 HTTP/1.1" 200 None 2026-04-04 05:04:06,635 - lava_callback - ERROR - Error processing callback for node 69cdf8b123eb053795d42ef6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54298 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54314 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:04:26,297 - lava_callback - INFO - Received callback for node 69cdf8d823eb053795d42f06 INFO: 172.20.0.1:38782 - "POST /node/69cdf8d823eb053795d42f06 HTTP/1.0" 202 Accepted 2026-04-04 05:04:26,302 - lava_callback - INFO - Starting async job submit for node 69cdf8d823eb053795d42f06 2026-04-04 05:04:26,303 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:04:26,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf8d823eb053795d42f06 HTTP/1.1" 200 None 2026-04-04 05:04:26,322 - lava_callback - DEBUG - Processing node 69cdf8d823eb053795d42f06: baseline-arm-baylibre 2026-04-04 05:04:26,322 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf8d823eb053795d42f06 2026-04-04 05:04:26,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:04:26,368 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf8d823eb053795d42f06 HTTP/1.1" 200 None 2026-04-04 05:04:26,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:04:26,382 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:04:26,384 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:04:26,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf8d823eb053795d42f06 HTTP/1.1" 200 None 2026-04-04 05:04:26,405 - lava_callback - ERROR - Error processing callback for node 69cdf8d823eb053795d42f06: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36008 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:05:06,335 - lava_callback - INFO - Received callback for node 69cdf8fb23eb053795d42f12 INFO: 172.20.0.1:34706 - "POST /node/69cdf8fb23eb053795d42f12 HTTP/1.0" 202 Accepted 2026-04-04 05:05:06,344 - lava_callback - INFO - Starting async job submit for node 69cdf8fb23eb053795d42f12 2026-04-04 05:05:06,345 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:05:06,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf8fb23eb053795d42f12 HTTP/1.1" 200 None 2026-04-04 05:05:06,375 - lava_callback - DEBUG - Processing node 69cdf8fb23eb053795d42f12: baseline-arm-baylibre 2026-04-04 05:05:06,375 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf8fb23eb053795d42f12 2026-04-04 05:05:06,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:05:06,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf8fb23eb053795d42f12 HTTP/1.1" 200 None 2026-04-04 05:05:06,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:05:06,408 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:05:06,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:05:06,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf8fb23eb053795d42f12 HTTP/1.1" 200 None 2026-04-04 05:05:06,443 - lava_callback - ERROR - Error processing callback for node 69cdf8fb23eb053795d42f12: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49144 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:05:46,393 - lava_callback - INFO - Received callback for node 69cdf92323eb053795d42f1c INFO: 172.20.0.1:33672 - "POST /node/69cdf92323eb053795d42f1c HTTP/1.0" 202 Accepted 2026-04-04 05:05:46,399 - lava_callback - INFO - Starting async job submit for node 69cdf92323eb053795d42f1c 2026-04-04 05:05:46,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:05:46,414 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf92323eb053795d42f1c HTTP/1.1" 200 None 2026-04-04 05:05:46,415 - lava_callback - DEBUG - Processing node 69cdf92323eb053795d42f1c: baseline-arm-baylibre 2026-04-04 05:05:46,415 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf92323eb053795d42f1c 2026-04-04 05:05:46,416 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:05:46,431 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf92323eb053795d42f1c HTTP/1.1" 200 None 2026-04-04 05:05:46,433 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:05:46,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:05:46,446 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:05:46,478 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf92323eb053795d42f1c HTTP/1.1" 200 None 2026-04-04 05:05:46,479 - lava_callback - ERROR - Error processing callback for node 69cdf92323eb053795d42f1c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59460 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59470 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:06:25,209 - lava_callback - INFO - Received callback for node 69cdf94a23eb053795d42f2c INFO: 172.20.0.1:49768 - "POST /node/69cdf94a23eb053795d42f2c HTTP/1.0" 202 Accepted 2026-04-04 05:06:25,214 - lava_callback - INFO - Starting async job submit for node 69cdf94a23eb053795d42f2c 2026-04-04 05:06:25,215 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:06:25,230 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf94a23eb053795d42f2c HTTP/1.1" 200 None 2026-04-04 05:06:25,231 - lava_callback - DEBUG - Processing node 69cdf94a23eb053795d42f2c: baseline-arm-clabbe 2026-04-04 05:06:25,231 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf94a23eb053795d42f2c 2026-04-04 05:06:25,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:06:25,245 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf94a23eb053795d42f2c HTTP/1.1" 200 None 2026-04-04 05:06:25,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:06:25,271 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:06:25,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:06:25,322 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf94a23eb053795d42f2c HTTP/1.1" 200 None 2026-04-04 05:06:25,329 - lava_callback - ERROR - Error processing callback for node 69cdf94a23eb053795d42f2c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45888 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:07:06,586 - lava_callback - INFO - Received callback for node 69cdf96b23eb053795d42f3d INFO: 172.20.0.1:45890 - "POST /node/69cdf96b23eb053795d42f3d HTTP/1.0" 202 Accepted 2026-04-04 05:07:06,592 - lava_callback - INFO - Starting async job submit for node 69cdf96b23eb053795d42f3d 2026-04-04 05:07:06,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:07:06,608 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf96b23eb053795d42f3d HTTP/1.1" 200 None 2026-04-04 05:07:06,609 - lava_callback - DEBUG - Processing node 69cdf96b23eb053795d42f3d: baseline-arm-baylibre 2026-04-04 05:07:06,610 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf96b23eb053795d42f3d 2026-04-04 05:07:06,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:07:06,630 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf96b23eb053795d42f3d HTTP/1.1" 200 None 2026-04-04 05:07:06,631 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:07:06,647 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:07:06,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:07:06,673 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf96b23eb053795d42f3d HTTP/1.1" 200 None 2026-04-04 05:07:06,674 - lava_callback - ERROR - Error processing callback for node 69cdf96b23eb053795d42f3d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54104 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54436 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:07:46,536 - lava_callback - INFO - Received callback for node 69cdf98e23eb053795d42f4a INFO: 172.20.0.1:54452 - "POST /node/69cdf98e23eb053795d42f4a HTTP/1.0" 202 Accepted 2026-04-04 05:07:46,541 - lava_callback - INFO - Starting async job submit for node 69cdf98e23eb053795d42f4a 2026-04-04 05:07:46,542 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:07:46,564 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf98e23eb053795d42f4a HTTP/1.1" 200 None 2026-04-04 05:07:46,565 - lava_callback - DEBUG - Processing node 69cdf98e23eb053795d42f4a: baseline-arm-baylibre 2026-04-04 05:07:46,565 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf98e23eb053795d42f4a 2026-04-04 05:07:46,566 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:07:46,578 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf98e23eb053795d42f4a HTTP/1.1" 200 None 2026-04-04 05:07:46,580 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:07:46,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:07:46,594 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:07:46,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf98e23eb053795d42f4a HTTP/1.1" 200 None 2026-04-04 05:07:46,619 - lava_callback - ERROR - Error processing callback for node 69cdf98e23eb053795d42f4a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38556 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38568 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:08:25,238 - lava_callback - INFO - Received callback for node 69cdf9b723eb053795d42f57 INFO: 172.20.0.1:46274 - "POST /node/69cdf9b723eb053795d42f57 HTTP/1.0" 202 Accepted 2026-04-04 05:08:25,244 - lava_callback - INFO - Starting async job submit for node 69cdf9b723eb053795d42f57 2026-04-04 05:08:25,245 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:08:25,264 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf9b723eb053795d42f57 HTTP/1.1" 200 None 2026-04-04 05:08:25,265 - lava_callback - DEBUG - Processing node 69cdf9b723eb053795d42f57: baseline-arm-clabbe 2026-04-04 05:08:25,265 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf9b723eb053795d42f57 2026-04-04 05:08:25,266 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:08:25,279 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf9b723eb053795d42f57 HTTP/1.1" 200 None 2026-04-04 05:08:25,280 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:08:25,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:08:25,294 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:08:25,314 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf9b723eb053795d42f57 HTTP/1.1" 200 None 2026-04-04 05:08:25,316 - lava_callback - ERROR - Error processing callback for node 69cdf9b723eb053795d42f57: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58042 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:08:46,598 - lava_callback - INFO - Received callback for node 69cdf9d923eb053795d42f63 INFO: 172.20.0.1:58048 - "POST /node/69cdf9d923eb053795d42f63 HTTP/1.0" 202 Accepted 2026-04-04 05:08:46,604 - lava_callback - INFO - Starting async job submit for node 69cdf9d923eb053795d42f63 2026-04-04 05:08:46,605 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:08:46,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf9d923eb053795d42f63 HTTP/1.1" 200 None 2026-04-04 05:08:46,619 - lava_callback - DEBUG - Processing node 69cdf9d923eb053795d42f63: baseline-arm-baylibre 2026-04-04 05:08:46,620 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf9d923eb053795d42f63 2026-04-04 05:08:46,620 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:08:46,635 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf9d923eb053795d42f63 HTTP/1.1" 200 None 2026-04-04 05:08:46,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:08:46,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:08:46,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:08:46,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf9d923eb053795d42f63 HTTP/1.1" 200 None 2026-04-04 05:08:46,672 - lava_callback - ERROR - Error processing callback for node 69cdf9d923eb053795d42f63: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41950 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37136 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:09:26,590 - lava_callback - INFO - Received callback for node 69cdf9fd23eb053795d42f70 INFO: 172.20.0.1:37146 - "POST /node/69cdf9fd23eb053795d42f70 HTTP/1.0" 202 Accepted 2026-04-04 05:09:26,596 - lava_callback - INFO - Starting async job submit for node 69cdf9fd23eb053795d42f70 2026-04-04 05:09:26,597 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:09:26,620 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf9fd23eb053795d42f70 HTTP/1.1" 200 None 2026-04-04 05:09:26,621 - lava_callback - DEBUG - Processing node 69cdf9fd23eb053795d42f70: baseline-arm-baylibre 2026-04-04 05:09:26,621 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdf9fd23eb053795d42f70 2026-04-04 05:09:26,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:09:26,634 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdf9fd23eb053795d42f70 HTTP/1.1" 200 None 2026-04-04 05:09:26,636 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:09:26,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:09:26,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:09:26,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdf9fd23eb053795d42f70 HTTP/1.1" 200 None 2026-04-04 05:09:26,671 - lava_callback - ERROR - Error processing callback for node 69cdf9fd23eb053795d42f70: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38558 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:10:06,668 - lava_callback - INFO - Received callback for node 69cdfa2223eb053795d42f85 INFO: 172.20.0.1:38572 - "POST /node/69cdfa2223eb053795d42f85 HTTP/1.0" 202 Accepted 2026-04-04 05:10:06,674 - lava_callback - INFO - Starting async job submit for node 69cdfa2223eb053795d42f85 2026-04-04 05:10:06,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:10:06,692 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfa2223eb053795d42f85 HTTP/1.1" 200 None 2026-04-04 05:10:06,692 - lava_callback - DEBUG - Processing node 69cdfa2223eb053795d42f85: baseline-arm-baylibre 2026-04-04 05:10:06,693 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfa2223eb053795d42f85 2026-04-04 05:10:06,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:10:06,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfa2223eb053795d42f85 HTTP/1.1" 200 None 2026-04-04 05:10:06,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:10:06,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:10:06,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:10:06,743 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfa2223eb053795d42f85 HTTP/1.1" 200 None 2026-04-04 05:10:06,745 - lava_callback - ERROR - Error processing callback for node 69cdfa2223eb053795d42f85: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45166 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45424 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:10:46,675 - lava_callback - INFO - Received callback for node 69cdfa4623eb053795d42f8e INFO: 172.20.0.1:45434 - "POST /node/69cdfa4623eb053795d42f8e HTTP/1.0" 202 Accepted 2026-04-04 05:10:46,680 - lava_callback - INFO - Starting async job submit for node 69cdfa4623eb053795d42f8e 2026-04-04 05:10:46,681 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:10:46,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfa4623eb053795d42f8e HTTP/1.1" 200 None 2026-04-04 05:10:46,697 - lava_callback - DEBUG - Processing node 69cdfa4623eb053795d42f8e: baseline-arm-baylibre 2026-04-04 05:10:46,697 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfa4623eb053795d42f8e 2026-04-04 05:10:46,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:10:46,711 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfa4623eb053795d42f8e HTTP/1.1" 200 None 2026-04-04 05:10:46,712 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:10:46,725 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:10:46,727 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:10:46,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfa4623eb053795d42f8e HTTP/1.1" 200 None 2026-04-04 05:10:46,752 - lava_callback - ERROR - Error processing callback for node 69cdfa4623eb053795d42f8e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56036 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50370 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:11:27,019 - lava_callback - INFO - Received callback for node 69cdfa6b23eb053795d42f9f INFO: 172.20.0.1:50374 - "POST /node/69cdfa6b23eb053795d42f9f HTTP/1.0" 202 Accepted 2026-04-04 05:11:27,024 - lava_callback - INFO - Starting async job submit for node 69cdfa6b23eb053795d42f9f 2026-04-04 05:11:27,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:11:27,040 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfa6b23eb053795d42f9f HTTP/1.1" 200 None 2026-04-04 05:11:27,040 - lava_callback - DEBUG - Processing node 69cdfa6b23eb053795d42f9f: baseline-arm-baylibre 2026-04-04 05:11:27,041 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfa6b23eb053795d42f9f 2026-04-04 05:11:27,041 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:11:27,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfa6b23eb053795d42f9f HTTP/1.1" 200 None 2026-04-04 05:11:27,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:11:27,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:11:27,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:11:27,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfa6b23eb053795d42f9f HTTP/1.1" 200 None 2026-04-04 05:11:27,117 - lava_callback - ERROR - Error processing callback for node 69cdfa6b23eb053795d42f9f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48130 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:12:05,354 - lava_callback - INFO - Received callback for node 69cdfa9323eb053795d42fac INFO: 172.20.0.1:44556 - "POST /node/69cdfa9323eb053795d42fac HTTP/1.0" 202 Accepted 2026-04-04 05:12:05,360 - lava_callback - INFO - Starting async job submit for node 69cdfa9323eb053795d42fac 2026-04-04 05:12:05,361 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:12:05,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfa9323eb053795d42fac HTTP/1.1" 200 None 2026-04-04 05:12:05,377 - lava_callback - DEBUG - Processing node 69cdfa9323eb053795d42fac: baseline-arm-clabbe 2026-04-04 05:12:05,377 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfa9323eb053795d42fac 2026-04-04 05:12:05,378 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:12:05,391 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfa9323eb053795d42fac HTTP/1.1" 200 None 2026-04-04 05:12:05,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:12:05,406 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:12:05,407 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:12:05,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfa9323eb053795d42fac HTTP/1.1" 200 None 2026-04-04 05:12:05,440 - lava_callback - ERROR - Error processing callback for node 69cdfa9323eb053795d42fac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57636 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33464 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37310 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:13:05,406 - lava_callback - INFO - Received callback for node 69cdfad323eb053795d42fbe INFO: 172.20.0.1:34288 - "POST /node/69cdfad323eb053795d42fbe HTTP/1.0" 202 Accepted 2026-04-04 05:13:05,411 - lava_callback - INFO - Starting async job submit for node 69cdfad323eb053795d42fbe 2026-04-04 05:13:05,412 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:13:05,426 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfad323eb053795d42fbe HTTP/1.1" 200 None 2026-04-04 05:13:05,427 - lava_callback - DEBUG - Processing node 69cdfad323eb053795d42fbe: baseline-arm-clabbe 2026-04-04 05:13:05,428 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfad323eb053795d42fbe 2026-04-04 05:13:05,428 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:13:05,442 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfad323eb053795d42fbe HTTP/1.1" 200 None 2026-04-04 05:13:05,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:13:05,457 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:13:05,459 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:13:05,481 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfad323eb053795d42fbe HTTP/1.1" 200 None 2026-04-04 05:13:05,486 - lava_callback - ERROR - Error processing callback for node 69cdfad323eb053795d42fbe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51944 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60772 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:13:46,846 - lava_callback - INFO - Received callback for node 69cdfaf623eb053795d42fcb INFO: 172.20.0.1:60776 - "POST /node/69cdfaf623eb053795d42fcb HTTP/1.0" 202 Accepted 2026-04-04 05:13:46,852 - lava_callback - INFO - Starting async job submit for node 69cdfaf623eb053795d42fcb 2026-04-04 05:13:46,853 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:13:46,867 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfaf623eb053795d42fcb HTTP/1.1" 200 None 2026-04-04 05:13:46,868 - lava_callback - DEBUG - Processing node 69cdfaf623eb053795d42fcb: baseline-arm-baylibre 2026-04-04 05:13:46,868 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfaf623eb053795d42fcb 2026-04-04 05:13:46,869 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:13:46,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfaf623eb053795d42fcb HTTP/1.1" 200 None 2026-04-04 05:13:46,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:13:46,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:13:46,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:13:46,931 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfaf623eb053795d42fcb HTTP/1.1" 200 None 2026-04-04 05:13:46,934 - lava_callback - ERROR - Error processing callback for node 69cdfaf623eb053795d42fcb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50174 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:14:25,458 - lava_callback - INFO - Received callback for node 69cdfb1c23eb053795d42fdb INFO: 172.20.0.1:55940 - "POST /node/69cdfb1c23eb053795d42fdb HTTP/1.0" 202 Accepted 2026-04-04 05:14:25,464 - lava_callback - INFO - Starting async job submit for node 69cdfb1c23eb053795d42fdb 2026-04-04 05:14:25,465 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:14:25,479 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfb1c23eb053795d42fdb HTTP/1.1" 200 None 2026-04-04 05:14:25,480 - lava_callback - DEBUG - Processing node 69cdfb1c23eb053795d42fdb: baseline-arm-clabbe 2026-04-04 05:14:25,480 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfb1c23eb053795d42fdb 2026-04-04 05:14:25,481 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:14:25,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfb1c23eb053795d42fdb HTTP/1.1" 200 None 2026-04-04 05:14:25,496 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:14:25,508 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:14:25,510 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:14:25,536 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfb1c23eb053795d42fdb HTTP/1.1" 200 None 2026-04-04 05:14:25,541 - lava_callback - ERROR - Error processing callback for node 69cdfb1c23eb053795d42fdb: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38154 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:14:45,451 - lava_callback - INFO - Received callback for node 69cdfb4223eb053795d42fe8 INFO: 172.20.0.1:44402 - "POST /node/69cdfb4223eb053795d42fe8 HTTP/1.0" 202 Accepted 2026-04-04 05:14:45,457 - lava_callback - INFO - Starting async job submit for node 69cdfb4223eb053795d42fe8 2026-04-04 05:14:45,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:14:45,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfb4223eb053795d42fe8 HTTP/1.1" 200 None 2026-04-04 05:14:45,475 - lava_callback - DEBUG - Processing node 69cdfb4223eb053795d42fe8: baseline-arm-clabbe 2026-04-04 05:14:45,475 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfb4223eb053795d42fe8 2026-04-04 05:14:45,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:14:45,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfb4223eb053795d42fe8 HTTP/1.1" 200 None 2026-04-04 05:14:45,490 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:14:45,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:14:45,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:14:45,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfb4223eb053795d42fe8 HTTP/1.1" 200 None 2026-04-04 05:14:45,526 - lava_callback - ERROR - Error processing callback for node 69cdfb4223eb053795d42fe8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44260 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38196 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59246 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47438 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:16:25,558 - lava_callback - INFO - Received callback for node 69cdfb9a23eb053795d42ffc INFO: 172.20.0.1:49264 - "POST /node/69cdfb9a23eb053795d42ffc HTTP/1.0" 202 Accepted 2026-04-04 05:16:25,564 - lava_callback - INFO - Starting async job submit for node 69cdfb9a23eb053795d42ffc 2026-04-04 05:16:25,565 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:16:25,580 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfb9a23eb053795d42ffc HTTP/1.1" 200 None 2026-04-04 05:16:25,581 - lava_callback - DEBUG - Processing node 69cdfb9a23eb053795d42ffc: baseline-arm-clabbe 2026-04-04 05:16:25,581 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfb9a23eb053795d42ffc 2026-04-04 05:16:25,581 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:16:25,594 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfb9a23eb053795d42ffc HTTP/1.1" 200 None 2026-04-04 05:16:25,596 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:16:25,609 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:16:25,610 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:16:25,631 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfb9a23eb053795d42ffc HTTP/1.1" 200 None 2026-04-04 05:16:25,637 - lava_callback - ERROR - Error processing callback for node 69cdfb9a23eb053795d42ffc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47756 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:17:05,547 - lava_callback - INFO - Received callback for node 69cdfbc123eb053795d43010 INFO: 172.20.0.1:37314 - "POST /node/69cdfbc123eb053795d43010 HTTP/1.0" 202 Accepted 2026-04-04 05:17:05,556 - lava_callback - INFO - Starting async job submit for node 69cdfbc123eb053795d43010 2026-04-04 05:17:05,557 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:17:05,572 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfbc123eb053795d43010 HTTP/1.1" 200 None 2026-04-04 05:17:05,573 - lava_callback - DEBUG - Processing node 69cdfbc123eb053795d43010: baseline-arm-clabbe 2026-04-04 05:17:05,573 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfbc123eb053795d43010 2026-04-04 05:17:05,574 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:17:05,592 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfbc123eb053795d43010 HTTP/1.1" 200 None 2026-04-04 05:17:05,593 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:17:05,613 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:17:05,614 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:17:05,652 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfbc123eb053795d43010 HTTP/1.1" 200 None 2026-04-04 05:17:05,653 - lava_callback - ERROR - Error processing callback for node 69cdfbc123eb053795d43010: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32884 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:18:05,650 - lava_callback - INFO - Received callback for node 69cdfbff23eb053795d43025 INFO: 172.20.0.1:46278 - "POST /node/69cdfbff23eb053795d43025 HTTP/1.0" 202 Accepted 2026-04-04 05:18:05,656 - lava_callback - INFO - Starting async job submit for node 69cdfbff23eb053795d43025 2026-04-04 05:18:05,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:18:05,670 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfbff23eb053795d43025 HTTP/1.1" 200 None 2026-04-04 05:18:05,671 - lava_callback - DEBUG - Processing node 69cdfbff23eb053795d43025: baseline-arm-clabbe 2026-04-04 05:18:05,671 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfbff23eb053795d43025 2026-04-04 05:18:05,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:18:05,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfbff23eb053795d43025 HTTP/1.1" 200 None 2026-04-04 05:18:05,686 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:18:05,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:18:05,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:18:05,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfbff23eb053795d43025 HTTP/1.1" 200 None 2026-04-04 05:18:05,723 - lava_callback - ERROR - Error processing callback for node 69cdfbff23eb053795d43025: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49652 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50978 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:18:45,628 - lava_callback - INFO - Received callback for node 69cdfc2523eb053795d43034 INFO: 172.20.0.1:33702 - "POST /node/69cdfc2523eb053795d43034 HTTP/1.0" 202 Accepted 2026-04-04 05:18:45,633 - lava_callback - INFO - Starting async job submit for node 69cdfc2523eb053795d43034 2026-04-04 05:18:45,634 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:18:45,648 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfc2523eb053795d43034 HTTP/1.1" 200 None 2026-04-04 05:18:45,649 - lava_callback - DEBUG - Processing node 69cdfc2523eb053795d43034: baseline-arm-clabbe 2026-04-04 05:18:45,649 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfc2523eb053795d43034 2026-04-04 05:18:45,649 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:18:45,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfc2523eb053795d43034 HTTP/1.1" 200 None 2026-04-04 05:18:45,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:18:45,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:18:45,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:18:45,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfc2523eb053795d43034 HTTP/1.1" 200 None 2026-04-04 05:18:45,707 - lava_callback - ERROR - Error processing callback for node 69cdfc2523eb053795d43034: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53472 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:20:05,837 - lava_callback - INFO - Received callback for node 69cdfc7823eb053795d4304d INFO: 172.20.0.1:53968 - "POST /node/69cdfc7823eb053795d4304d HTTP/1.0" 202 Accepted 2026-04-04 05:20:05,846 - lava_callback - INFO - Starting async job submit for node 69cdfc7823eb053795d4304d 2026-04-04 05:20:05,848 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:20:05,876 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfc7823eb053795d4304d HTTP/1.1" 200 None 2026-04-04 05:20:05,878 - lava_callback - DEBUG - Processing node 69cdfc7823eb053795d4304d: baseline-arm-clabbe 2026-04-04 05:20:05,878 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfc7823eb053795d4304d 2026-04-04 05:20:05,879 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:20:05,893 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfc7823eb053795d4304d HTTP/1.1" 200 None 2026-04-04 05:20:05,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:20:05,907 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:20:05,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:20:05,930 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfc7823eb053795d4304d HTTP/1.1" 200 None 2026-04-04 05:20:05,931 - lava_callback - ERROR - Error processing callback for node 69cdfc7823eb053795d4304d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53970 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58366 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58378 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37868 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:20:45,706 - lava_callback - INFO - Received callback for node 69cdfc9e23eb053795d43056 INFO: 172.20.0.1:34992 - "POST /node/69cdfc9e23eb053795d43056 HTTP/1.0" 202 Accepted 2026-04-04 05:20:45,712 - lava_callback - INFO - Starting async job submit for node 69cdfc9e23eb053795d43056 2026-04-04 05:20:45,713 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:20:45,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfc9e23eb053795d43056 HTTP/1.1" 200 None 2026-04-04 05:20:45,728 - lava_callback - DEBUG - Processing node 69cdfc9e23eb053795d43056: baseline-arm-clabbe 2026-04-04 05:20:45,728 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfc9e23eb053795d43056 2026-04-04 05:20:45,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:20:45,742 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfc9e23eb053795d43056 HTTP/1.1" 200 None 2026-04-04 05:20:45,744 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:20:45,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:20:45,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:20:45,780 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfc9e23eb053795d43056 HTTP/1.1" 200 None 2026-04-04 05:20:45,782 - lava_callback - ERROR - Error processing callback for node 69cdfc9e23eb053795d43056: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35008 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35044 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:21:25,784 - lava_callback - INFO - Received callback for node 69cdfcc223eb053795d43069 INFO: 172.20.0.1:47888 - "POST /node/69cdfcc223eb053795d43069 HTTP/1.0" 202 Accepted 2026-04-04 05:21:25,789 - lava_callback - INFO - Starting async job submit for node 69cdfcc223eb053795d43069 2026-04-04 05:21:25,790 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:21:25,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfcc223eb053795d43069 HTTP/1.1" 200 None 2026-04-04 05:21:25,806 - lava_callback - DEBUG - Processing node 69cdfcc223eb053795d43069: baseline-arm-clabbe 2026-04-04 05:21:25,806 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfcc223eb053795d43069 2026-04-04 05:21:25,807 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:21:25,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfcc223eb053795d43069 HTTP/1.1" 200 None 2026-04-04 05:21:25,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:21:25,834 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:21:25,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:21:25,863 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfcc223eb053795d43069 HTTP/1.1" 200 None 2026-04-04 05:21:25,866 - lava_callback - ERROR - Error processing callback for node 69cdfcc223eb053795d43069: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48334 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:21:45,745 - lava_callback - INFO - Received callback for node 69cdfce723eb053795d43077 INFO: 172.20.0.1:59076 - "POST /node/69cdfce723eb053795d43077 HTTP/1.0" 202 Accepted 2026-04-04 05:21:45,750 - lava_callback - INFO - Starting async job submit for node 69cdfce723eb053795d43077 2026-04-04 05:21:45,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:21:45,789 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfce723eb053795d43077 HTTP/1.1" 200 None 2026-04-04 05:21:45,790 - lava_callback - DEBUG - Processing node 69cdfce723eb053795d43077: baseline-arm-clabbe 2026-04-04 05:21:45,790 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfce723eb053795d43077 2026-04-04 05:21:45,791 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:21:45,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfce723eb053795d43077 HTTP/1.1" 200 None 2026-04-04 05:21:45,806 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:21:45,994 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:21:45,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:21:46,016 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfce723eb053795d43077 HTTP/1.1" 200 None 2026-04-04 05:21:46,017 - lava_callback - ERROR - Error processing callback for node 69cdfce723eb053795d43077: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39782 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39796 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34818 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33572 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:23:05,885 - lava_callback - INFO - Received callback for node 69cdfd2c23eb053795d43092 INFO: 172.20.0.1:40700 - "POST /node/69cdfd2c23eb053795d43092 HTTP/1.0" 202 Accepted 2026-04-04 05:23:05,890 - lava_callback - INFO - Starting async job submit for node 69cdfd2c23eb053795d43092 2026-04-04 05:23:05,891 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:23:05,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfd2c23eb053795d43092 HTTP/1.1" 200 None 2026-04-04 05:23:05,909 - lava_callback - DEBUG - Processing node 69cdfd2c23eb053795d43092: baseline-arm-clabbe 2026-04-04 05:23:05,909 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfd2c23eb053795d43092 2026-04-04 05:23:05,910 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:23:05,923 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfd2c23eb053795d43092 HTTP/1.1" 200 None 2026-04-04 05:23:05,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:23:05,937 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa2484b5e3e0ab6a60b57 HTTP/1.1" 200 None 2026-04-04 05:23:05,939 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:23:05,958 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfd2c23eb053795d43092 HTTP/1.1" 200 None 2026-04-04 05:23:05,960 - lava_callback - ERROR - Error processing callback for node 69cdfd2c23eb053795d43092: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41602 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52762 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52774 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42282 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57344 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50158 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60582 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39680 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:29:07,778 - lava_callback - INFO - Received callback for node 69cdfe9c23eb053795d430a8 INFO: 172.20.0.1:39682 - "POST /node/69cdfe9c23eb053795d430a8 HTTP/1.0" 202 Accepted 2026-04-04 05:29:07,784 - lava_callback - INFO - Starting async job submit for node 69cdfe9c23eb053795d430a8 2026-04-04 05:29:07,785 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:29:07,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfe9c23eb053795d430a8 HTTP/1.1" 200 None 2026-04-04 05:29:07,800 - lava_callback - DEBUG - Processing node 69cdfe9c23eb053795d430a8: baseline-arm64-baylibre 2026-04-04 05:29:07,800 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdfe9c23eb053795d430a8 2026-04-04 05:29:07,801 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:29:07,820 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdfe9c23eb053795d430a8 HTTP/1.1" 200 None 2026-04-04 05:29:07,821 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:29:07,835 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:29:07,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:29:07,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdfe9c23eb053795d430a8 HTTP/1.1" 200 None 2026-04-04 05:29:07,861 - lava_callback - ERROR - Error processing callback for node 69cdfe9c23eb053795d430a8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56656 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56506 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58694 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36280 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:32:07,998 - lava_callback - INFO - Received callback for node 69cdff5123eb053795d430b7 INFO: 172.20.0.1:36290 - "POST /node/69cdff5123eb053795d430b7 HTTP/1.0" 202 Accepted 2026-04-04 05:32:08,006 - lava_callback - INFO - Starting async job submit for node 69cdff5123eb053795d430b7 2026-04-04 05:32:08,006 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:32:08,022 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdff5123eb053795d430b7 HTTP/1.1" 200 None 2026-04-04 05:32:08,023 - lava_callback - DEBUG - Processing node 69cdff5123eb053795d430b7: baseline-arm64-baylibre 2026-04-04 05:32:08,023 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69cdff5123eb053795d430b7 2026-04-04 05:32:08,023 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:32:08,047 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdff5123eb053795d430b7 HTTP/1.1" 200 None 2026-04-04 05:32:08,049 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:32:08,066 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:32:08,068 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:32:08,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69cdff5123eb053795d430b7 HTTP/1.1" 200 None 2026-04-04 05:32:08,093 - lava_callback - ERROR - Error processing callback for node 69cdff5123eb053795d430b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47778 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59114 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35746 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:35:08,083 - lava_callback - INFO - Received callback for node 69ce000523eb053795d430c8 INFO: 172.20.0.1:35748 - "POST /node/69ce000523eb053795d430c8 HTTP/1.0" 202 Accepted 2026-04-04 05:35:08,090 - lava_callback - INFO - Starting async job submit for node 69ce000523eb053795d430c8 2026-04-04 05:35:08,091 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:35:08,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce000523eb053795d430c8 HTTP/1.1" 200 None 2026-04-04 05:35:08,106 - lava_callback - DEBUG - Processing node 69ce000523eb053795d430c8: baseline-arm64-baylibre 2026-04-04 05:35:08,107 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce000523eb053795d430c8 2026-04-04 05:35:08,107 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:35:08,122 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce000523eb053795d430c8 HTTP/1.1" 200 None 2026-04-04 05:35:08,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:35:08,139 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:35:08,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:35:08,164 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce000523eb053795d430c8 HTTP/1.1" 200 None 2026-04-04 05:35:08,166 - lava_callback - ERROR - Error processing callback for node 69ce000523eb053795d430c8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47626 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41202 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52134 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57232 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46516 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:37:26,462 - lava_callback - INFO - Received callback for node 69ce008323eb053795d430d5 INFO: 172.20.0.1:51362 - "POST /node/69ce008323eb053795d430d5 HTTP/1.0" 202 Accepted 2026-04-04 05:37:26,468 - lava_callback - INFO - Starting async job submit for node 69ce008323eb053795d430d5 2026-04-04 05:37:26,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:37:26,483 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce008323eb053795d430d5 HTTP/1.1" 200 None 2026-04-04 05:37:26,484 - lava_callback - DEBUG - Processing node 69ce008323eb053795d430d5: baseline-arm64-clabbe 2026-04-04 05:37:26,484 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce008323eb053795d430d5 2026-04-04 05:37:26,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:37:26,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce008323eb053795d430d5 HTTP/1.1" 200 None 2026-04-04 05:37:26,499 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:37:26,515 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:37:26,516 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:51366 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:37:26,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce008323eb053795d430d5 HTTP/1.1" 200 None 2026-04-04 05:37:26,539 - lava_callback - ERROR - Error processing callback for node 69ce008323eb053795d430d5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56304 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35936 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52100 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52108 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:39:26,488 - lava_callback - INFO - Received callback for node 69ce00fc23eb053795d430e4 INFO: 172.20.0.1:45204 - "POST /node/69ce00fc23eb053795d430e4 HTTP/1.0" 202 Accepted 2026-04-04 05:39:26,494 - lava_callback - INFO - Starting async job submit for node 69ce00fc23eb053795d430e4 2026-04-04 05:39:26,494 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:39:26,509 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce00fc23eb053795d430e4 HTTP/1.1" 200 None 2026-04-04 05:39:26,510 - lava_callback - DEBUG - Processing node 69ce00fc23eb053795d430e4: baseline-arm64-clabbe 2026-04-04 05:39:26,510 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce00fc23eb053795d430e4 2026-04-04 05:39:26,511 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:45214 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:39:26,524 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce00fc23eb053795d430e4 HTTP/1.1" 200 None 2026-04-04 05:39:26,526 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:39:26,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:39:26,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:39:26,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce00fc23eb053795d430e4 HTTP/1.1" 200 None 2026-04-04 05:39:26,571 - lava_callback - ERROR - Error processing callback for node 69ce00fc23eb053795d430e4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35010 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37776 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53536 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53548 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39470 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60770 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:42:26,674 - lava_callback - INFO - Received callback for node 69ce01b023eb053795d430f3 INFO: 172.20.0.1:60786 - "POST /node/69ce01b023eb053795d430f3 HTTP/1.0" 202 Accepted 2026-04-04 05:42:26,680 - lava_callback - INFO - Starting async job submit for node 69ce01b023eb053795d430f3 2026-04-04 05:42:26,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:42:26,695 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce01b023eb053795d430f3 HTTP/1.1" 200 None 2026-04-04 05:42:26,696 - lava_callback - DEBUG - Processing node 69ce01b023eb053795d430f3: baseline-arm64-clabbe 2026-04-04 05:42:26,696 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce01b023eb053795d430f3 2026-04-04 05:42:26,697 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:42:26,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce01b023eb053795d430f3 HTTP/1.1" 200 None 2026-04-04 05:42:26,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:42:26,762 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:42:26,763 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:42:26,784 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce01b023eb053795d430f3 HTTP/1.1" 200 None 2026-04-04 05:42:26,785 - lava_callback - ERROR - Error processing callback for node 69ce01b023eb053795d430f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43592 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36922 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58710 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:44:26,736 - lava_callback - INFO - Received callback for node 69ce022823eb053795d43101 INFO: 172.20.0.1:58714 - "POST /node/69ce022823eb053795d43101 HTTP/1.0" 202 Accepted 2026-04-04 05:44:26,741 - lava_callback - INFO - Starting async job submit for node 69ce022823eb053795d43101 2026-04-04 05:44:26,742 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:44:26,756 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce022823eb053795d43101 HTTP/1.1" 200 None 2026-04-04 05:44:26,757 - lava_callback - DEBUG - Processing node 69ce022823eb053795d43101: baseline-arm64-clabbe 2026-04-04 05:44:26,757 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce022823eb053795d43101 2026-04-04 05:44:26,758 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:44:26,772 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce022823eb053795d43101 HTTP/1.1" 200 None 2026-04-04 05:44:26,774 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:44:26,799 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:44:26,800 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:44:26,821 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce022823eb053795d43101 HTTP/1.1" 200 None 2026-04-04 05:44:26,825 - lava_callback - ERROR - Error processing callback for node 69ce022823eb053795d43101: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41472 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53114 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:45:46,765 - lava_callback - INFO - Received callback for node 69ce027523eb053795d4310d INFO: 172.20.0.1:53116 - "POST /node/69ce027523eb053795d4310d HTTP/1.0" 202 Accepted 2026-04-04 05:45:46,770 - lava_callback - INFO - Starting async job submit for node 69ce027523eb053795d4310d 2026-04-04 05:45:46,771 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:45:46,791 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce027523eb053795d4310d HTTP/1.1" 200 None 2026-04-04 05:45:46,792 - lava_callback - DEBUG - Processing node 69ce027523eb053795d4310d: baseline-arm64-clabbe 2026-04-04 05:45:46,792 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce027523eb053795d4310d 2026-04-04 05:45:46,793 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:45:46,807 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce027523eb053795d4310d HTTP/1.1" 200 None 2026-04-04 05:45:46,809 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:45:46,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:45:46,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:45:46,845 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce027523eb053795d4310d HTTP/1.1" 200 None 2026-04-04 05:45:46,846 - lava_callback - ERROR - Error processing callback for node 69ce027523eb053795d4310d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48222 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48236 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35228 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50348 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39770 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:47:26,822 - lava_callback - INFO - Received callback for node 69ce02dc23eb053795d43118 INFO: 172.20.0.1:39776 - "POST /node/69ce02dc23eb053795d43118 HTTP/1.0" 202 Accepted 2026-04-04 05:47:26,828 - lava_callback - INFO - Starting async job submit for node 69ce02dc23eb053795d43118 2026-04-04 05:47:26,828 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:47:26,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce02dc23eb053795d43118 HTTP/1.1" 200 None 2026-04-04 05:47:26,843 - lava_callback - DEBUG - Processing node 69ce02dc23eb053795d43118: baseline-arm64-clabbe 2026-04-04 05:47:26,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce02dc23eb053795d43118 2026-04-04 05:47:26,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:47:26,857 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce02dc23eb053795d43118 HTTP/1.1" 200 None 2026-04-04 05:47:26,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:47:26,883 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:47:26,884 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:47:26,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce02dc23eb053795d43118 HTTP/1.1" 200 None 2026-04-04 05:47:26,911 - lava_callback - ERROR - Error processing callback for node 69ce02dc23eb053795d43118: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41586 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52660 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38762 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:48:28,771 - lava_callback - INFO - Received callback for node 69ce032023eb053795d43121 INFO: 172.20.0.1:38772 - "POST /node/69ce032023eb053795d43121 HTTP/1.0" 202 Accepted 2026-04-04 05:48:28,777 - lava_callback - INFO - Starting async job submit for node 69ce032023eb053795d43121 2026-04-04 05:48:28,778 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:48:28,798 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce032023eb053795d43121 HTTP/1.1" 200 None 2026-04-04 05:48:28,799 - lava_callback - DEBUG - Processing node 69ce032023eb053795d43121: baseline-arm64-baylibre 2026-04-04 05:48:28,799 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce032023eb053795d43121 2026-04-04 05:48:28,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:48:28,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce032023eb053795d43121 HTTP/1.1" 200 None 2026-04-04 05:48:28,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:48:28,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:48:28,839 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:48:28,860 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce032023eb053795d43121 HTTP/1.1" 200 None 2026-04-04 05:48:28,870 - lava_callback - ERROR - Error processing callback for node 69ce032023eb053795d43121: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32998 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58416 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49572 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:49:49,002 - lava_callback - INFO - Received callback for node 69ce036c23eb053795d4312f INFO: 172.20.0.1:49580 - "POST /node/69ce036c23eb053795d4312f HTTP/1.0" 202 Accepted 2026-04-04 05:49:49,008 - lava_callback - INFO - Starting async job submit for node 69ce036c23eb053795d4312f 2026-04-04 05:49:49,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:49:49,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce036c23eb053795d4312f HTTP/1.1" 200 None 2026-04-04 05:49:49,024 - lava_callback - DEBUG - Processing node 69ce036c23eb053795d4312f: baseline-arm64-baylibre 2026-04-04 05:49:49,024 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce036c23eb053795d4312f 2026-04-04 05:49:49,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:49:49,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce036c23eb053795d4312f HTTP/1.1" 200 None 2026-04-04 05:49:49,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:49:49,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:49:49,053 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:49:49,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce036c23eb053795d4312f HTTP/1.1" 200 None 2026-04-04 05:49:49,079 - lava_callback - ERROR - Error processing callback for node 69ce036c23eb053795d4312f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56956 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46106 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33476 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39272 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43124 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:51:08,898 - lava_callback - INFO - Received callback for node 69ce03b723eb053795d43139 INFO: 172.20.0.1:43128 - "POST /node/69ce03b723eb053795d43139 HTTP/1.0" 202 Accepted 2026-04-04 05:51:08,903 - lava_callback - INFO - Starting async job submit for node 69ce03b723eb053795d43139 2026-04-04 05:51:08,904 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:51:08,918 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce03b723eb053795d43139 HTTP/1.1" 200 None 2026-04-04 05:51:08,919 - lava_callback - DEBUG - Processing node 69ce03b723eb053795d43139: baseline-arm64-baylibre 2026-04-04 05:51:08,919 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce03b723eb053795d43139 2026-04-04 05:51:08,920 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:51:08,933 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce03b723eb053795d43139 HTTP/1.1" 200 None 2026-04-04 05:51:08,934 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:51:08,956 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:51:08,957 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:51:08,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce03b723eb053795d43139 HTTP/1.1" 200 None 2026-04-04 05:51:08,982 - lava_callback - ERROR - Error processing callback for node 69ce03b723eb053795d43139: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59882 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59898 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44526 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38920 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38926 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44890 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:52:27,095 - lava_callback - INFO - Received callback for node 69ce040823eb053795d43149 INFO: 172.20.0.1:44898 - "POST /node/69ce040823eb053795d43149 HTTP/1.0" 202 Accepted 2026-04-04 05:52:27,100 - lava_callback - INFO - Starting async job submit for node 69ce040823eb053795d43149 2026-04-04 05:52:27,101 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:52:27,116 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce040823eb053795d43149 HTTP/1.1" 200 None 2026-04-04 05:52:27,117 - lava_callback - DEBUG - Processing node 69ce040823eb053795d43149: baseline-arm64-clabbe 2026-04-04 05:52:27,117 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce040823eb053795d43149 2026-04-04 05:52:27,117 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:52:27,130 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce040823eb053795d43149 HTTP/1.1" 200 None 2026-04-04 05:52:27,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:52:27,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:52:27,149 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:52:27,181 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce040823eb053795d43149 HTTP/1.1" 200 None 2026-04-04 05:52:27,183 - lava_callback - ERROR - Error processing callback for node 69ce040823eb053795d43149: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46240 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35030 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:53:27,106 - lava_callback - INFO - Received callback for node 69ce045223eb053795d43155 INFO: 172.20.0.1:35040 - "POST /node/69ce045223eb053795d43155 HTTP/1.0" 202 Accepted 2026-04-04 05:53:27,111 - lava_callback - INFO - Starting async job submit for node 69ce045223eb053795d43155 2026-04-04 05:53:27,112 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:53:27,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce045223eb053795d43155 HTTP/1.1" 200 None 2026-04-04 05:53:27,138 - lava_callback - DEBUG - Processing node 69ce045223eb053795d43155: baseline-arm64-clabbe 2026-04-04 05:53:27,138 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce045223eb053795d43155 2026-04-04 05:53:27,139 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:53:27,152 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce045223eb053795d43155 HTTP/1.1" 200 None 2026-04-04 05:53:27,154 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:53:27,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:53:27,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:53:27,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce045223eb053795d43155 HTTP/1.1" 200 None 2026-04-04 05:53:27,198 - lava_callback - ERROR - Error processing callback for node 69ce045223eb053795d43155: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55414 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55418 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38508 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:54:47,159 - lava_callback - INFO - Received callback for node 69ce049c23eb053795d43166 INFO: 172.20.0.1:38524 - "POST /node/69ce049c23eb053795d43166 HTTP/1.0" 202 Accepted 2026-04-04 05:54:47,164 - lava_callback - INFO - Starting async job submit for node 69ce049c23eb053795d43166 2026-04-04 05:54:47,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:54:47,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce049c23eb053795d43166 HTTP/1.1" 200 None 2026-04-04 05:54:47,190 - lava_callback - DEBUG - Processing node 69ce049c23eb053795d43166: baseline-arm64-clabbe 2026-04-04 05:54:47,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce049c23eb053795d43166 2026-04-04 05:54:47,192 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:54:47,206 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce049c23eb053795d43166 HTTP/1.1" 200 None 2026-04-04 05:54:47,208 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:54:47,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:54:47,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:54:47,255 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce049c23eb053795d43166 HTTP/1.1" 200 None 2026-04-04 05:54:47,256 - lava_callback - ERROR - Error processing callback for node 69ce049c23eb053795d43166: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48624 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54154 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41620 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:55:49,311 - lava_callback - INFO - Received callback for node 69ce04de23eb053795d43172 INFO: 172.20.0.1:41634 - "POST /node/69ce04de23eb053795d43172 HTTP/1.0" 202 Accepted 2026-04-04 05:55:49,317 - lava_callback - INFO - Starting async job submit for node 69ce04de23eb053795d43172 2026-04-04 05:55:49,318 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:55:49,332 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce04de23eb053795d43172 HTTP/1.1" 200 None 2026-04-04 05:55:49,333 - lava_callback - DEBUG - Processing node 69ce04de23eb053795d43172: baseline-arm64-baylibre 2026-04-04 05:55:49,333 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce04de23eb053795d43172 2026-04-04 05:55:49,334 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:55:49,349 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce04de23eb053795d43172 HTTP/1.1" 200 None 2026-04-04 05:55:49,350 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:55:49,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:55:49,376 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:55:49,396 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce04de23eb053795d43172 HTTP/1.1" 200 None 2026-04-04 05:55:49,397 - lava_callback - ERROR - Error processing callback for node 69ce04de23eb053795d43172: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43316 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38412 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45102 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:57:07,242 - lava_callback - INFO - Received callback for node 69ce053023eb053795d43183 INFO: 172.20.0.1:45106 - "POST /node/69ce053023eb053795d43183 HTTP/1.0" 202 Accepted 2026-04-04 05:57:07,250 - lava_callback - INFO - Starting async job submit for node 69ce053023eb053795d43183 2026-04-04 05:57:07,252 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:57:07,266 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce053023eb053795d43183 HTTP/1.1" 200 None 2026-04-04 05:57:07,267 - lava_callback - DEBUG - Processing node 69ce053023eb053795d43183: baseline-arm64-clabbe 2026-04-04 05:57:07,267 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce053023eb053795d43183 2026-04-04 05:57:07,268 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:57:07,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce053023eb053795d43183 HTTP/1.1" 200 None 2026-04-04 05:57:07,283 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:57:07,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:57:07,301 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:57:07,328 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce053023eb053795d43183 HTTP/1.1" 200 None 2026-04-04 05:57:07,329 - lava_callback - ERROR - Error processing callback for node 69ce053023eb053795d43183: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37264 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49644 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49654 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49660 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47956 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:58:29,302 - lava_callback - INFO - Received callback for node 69ce057523eb053795d4318d INFO: 172.20.0.1:47962 - "POST /node/69ce057523eb053795d4318d HTTP/1.0" 202 Accepted 2026-04-04 05:58:29,307 - lava_callback - INFO - Starting async job submit for node 69ce057523eb053795d4318d 2026-04-04 05:58:29,308 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:58:29,323 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce057523eb053795d4318d HTTP/1.1" 200 None 2026-04-04 05:58:29,323 - lava_callback - DEBUG - Processing node 69ce057523eb053795d4318d: baseline-arm64-baylibre 2026-04-04 05:58:29,324 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce057523eb053795d4318d 2026-04-04 05:58:29,324 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:58:29,338 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce057523eb053795d4318d HTTP/1.1" 200 None 2026-04-04 05:58:29,339 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:58:29,352 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:58:29,354 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:58:29,374 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce057523eb053795d4318d HTTP/1.1" 200 None 2026-04-04 05:58:29,375 - lava_callback - ERROR - Error processing callback for node 69ce057523eb053795d4318d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37362 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34940 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34954 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34960 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48346 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38154 - "GET / HTTP/1.0" 200 OK 2026-04-04 05:59:47,382 - lava_callback - INFO - Received callback for node 69ce05c623eb053795d4319d INFO: 172.20.0.1:38156 - "POST /node/69ce05c623eb053795d4319d HTTP/1.0" 202 Accepted 2026-04-04 05:59:47,388 - lava_callback - INFO - Starting async job submit for node 69ce05c623eb053795d4319d 2026-04-04 05:59:47,389 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:59:47,404 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce05c623eb053795d4319d HTTP/1.1" 200 None 2026-04-04 05:59:47,404 - lava_callback - DEBUG - Processing node 69ce05c623eb053795d4319d: baseline-arm64-clabbe 2026-04-04 05:59:47,405 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce05c623eb053795d4319d 2026-04-04 05:59:47,405 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:59:47,418 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce05c623eb053795d4319d HTTP/1.1" 200 None 2026-04-04 05:59:47,419 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:59:47,433 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 05:59:47,434 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 05:59:47,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce05c623eb053795d4319d HTTP/1.1" 200 None 2026-04-04 05:59:47,455 - lava_callback - ERROR - Error processing callback for node 69ce05c623eb053795d4319d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58516 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58526 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56220 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:01:07,440 - lava_callback - INFO - Received callback for node 69ce061423eb053795d431a7 INFO: 172.20.0.1:56224 - "POST /node/69ce061423eb053795d431a7 HTTP/1.0" 202 Accepted 2026-04-04 06:01:07,446 - lava_callback - INFO - Starting async job submit for node 69ce061423eb053795d431a7 2026-04-04 06:01:07,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:01:07,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce061423eb053795d431a7 HTTP/1.1" 200 None 2026-04-04 06:01:07,461 - lava_callback - DEBUG - Processing node 69ce061423eb053795d431a7: baseline-arm64-clabbe 2026-04-04 06:01:07,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce061423eb053795d431a7 2026-04-04 06:01:07,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:01:07,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce061423eb053795d431a7 HTTP/1.1" 200 None 2026-04-04 06:01:07,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:01:07,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 06:01:07,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:01:07,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce061423eb053795d431a7 HTTP/1.1" 200 None 2026-04-04 06:01:07,513 - lava_callback - ERROR - Error processing callback for node 69ce061423eb053795d431a7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47852 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33372 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50522 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:02:09,516 - lava_callback - INFO - Received callback for node 69ce065923eb053795d431b5 INFO: 172.20.0.1:50532 - "POST /node/69ce065923eb053795d431b5 HTTP/1.0" 202 Accepted 2026-04-04 06:02:09,522 - lava_callback - INFO - Starting async job submit for node 69ce065923eb053795d431b5 2026-04-04 06:02:09,523 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:02:09,538 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce065923eb053795d431b5 HTTP/1.1" 200 None 2026-04-04 06:02:09,539 - lava_callback - DEBUG - Processing node 69ce065923eb053795d431b5: baseline-arm64-baylibre 2026-04-04 06:02:09,539 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce065923eb053795d431b5 2026-04-04 06:02:09,539 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:02:09,583 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce065923eb053795d431b5 HTTP/1.1" 200 None 2026-04-04 06:02:09,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:02:09,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 06:02:09,599 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:02:09,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce065923eb053795d431b5 HTTP/1.1" 200 None 2026-04-04 06:02:09,645 - lava_callback - ERROR - Error processing callback for node 69ce065923eb053795d431b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35750 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52070 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49822 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49836 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40718 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:03:27,482 - lava_callback - INFO - Received callback for node 69ce06aa23eb053795d431c1 INFO: 172.20.0.1:40726 - "POST /node/69ce06aa23eb053795d431c1 HTTP/1.0" 202 Accepted 2026-04-04 06:03:27,488 - lava_callback - INFO - Starting async job submit for node 69ce06aa23eb053795d431c1 2026-04-04 06:03:27,488 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:03:27,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce06aa23eb053795d431c1 HTTP/1.1" 200 None 2026-04-04 06:03:27,504 - lava_callback - DEBUG - Processing node 69ce06aa23eb053795d431c1: baseline-arm64-clabbe 2026-04-04 06:03:27,504 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce06aa23eb053795d431c1 2026-04-04 06:03:27,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:03:27,518 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce06aa23eb053795d431c1 HTTP/1.1" 200 None 2026-04-04 06:03:27,520 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:03:27,533 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 06:03:27,534 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:03:27,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce06aa23eb053795d431c1 HTTP/1.1" 200 None 2026-04-04 06:03:27,555 - lava_callback - ERROR - Error processing callback for node 69ce06aa23eb053795d431c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57686 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60716 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40020 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54760 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:04:47,541 - lava_callback - INFO - Received callback for node 69ce06f723eb053795d431d1 INFO: 172.20.0.1:54772 - "POST /node/69ce06f723eb053795d431d1 HTTP/1.0" 202 Accepted 2026-04-04 06:04:47,546 - lava_callback - INFO - Starting async job submit for node 69ce06f723eb053795d431d1 2026-04-04 06:04:47,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:04:47,562 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce06f723eb053795d431d1 HTTP/1.1" 200 None 2026-04-04 06:04:47,563 - lava_callback - DEBUG - Processing node 69ce06f723eb053795d431d1: baseline-arm64-clabbe 2026-04-04 06:04:47,563 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce06f723eb053795d431d1 2026-04-04 06:04:47,564 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:04:47,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce06f723eb053795d431d1 HTTP/1.1" 200 None 2026-04-04 06:04:47,578 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:04:47,590 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 06:04:47,592 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:04:47,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce06f723eb053795d431d1 HTTP/1.1" 200 None 2026-04-04 06:04:47,616 - lava_callback - ERROR - Error processing callback for node 69ce06f723eb053795d431d1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60574 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54056 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35034 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34116 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44028 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53152 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39936 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:08:07,744 - lava_callback - INFO - Received callback for node 69ce07b323eb053795d431e1 INFO: 172.20.0.1:39944 - "POST /node/69ce07b323eb053795d431e1 HTTP/1.0" 202 Accepted 2026-04-04 06:08:07,751 - lava_callback - INFO - Starting async job submit for node 69ce07b323eb053795d431e1 2026-04-04 06:08:07,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:08:07,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce07b323eb053795d431e1 HTTP/1.1" 200 None 2026-04-04 06:08:07,767 - lava_callback - DEBUG - Processing node 69ce07b323eb053795d431e1: baseline-arm64-clabbe 2026-04-04 06:08:07,767 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce07b323eb053795d431e1 2026-04-04 06:08:07,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:08:07,783 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce07b323eb053795d431e1 HTTP/1.1" 200 None 2026-04-04 06:08:07,784 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:08:07,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 06:08:07,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:08:07,826 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce07b323eb053795d431e1 HTTP/1.1" 200 None 2026-04-04 06:08:07,841 - lava_callback - ERROR - Error processing callback for node 69ce07b323eb053795d431e1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57544 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45724 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45726 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54066 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:09:27,783 - lava_callback - INFO - Received callback for node 69ce080223eb053795d431f2 INFO: 172.20.0.1:54076 - "POST /node/69ce080223eb053795d431f2 HTTP/1.0" 202 Accepted 2026-04-04 06:09:27,789 - lava_callback - INFO - Starting async job submit for node 69ce080223eb053795d431f2 2026-04-04 06:09:27,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:09:27,804 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce080223eb053795d431f2 HTTP/1.1" 200 None 2026-04-04 06:09:27,804 - lava_callback - DEBUG - Processing node 69ce080223eb053795d431f2: baseline-arm64-clabbe 2026-04-04 06:09:27,805 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce080223eb053795d431f2 2026-04-04 06:09:27,805 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:09:27,822 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce080223eb053795d431f2 HTTP/1.1" 200 None 2026-04-04 06:09:27,824 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:09:27,837 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 06:09:27,838 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:09:27,858 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce080223eb053795d431f2 HTTP/1.1" 200 None 2026-04-04 06:09:27,859 - lava_callback - ERROR - Error processing callback for node 69ce080223eb053795d431f2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47566 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33578 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51508 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:10:47,818 - lava_callback - INFO - Received callback for node 69ce085223eb053795d43204 INFO: 172.20.0.1:51516 - "POST /node/69ce085223eb053795d43204 HTTP/1.0" 202 Accepted 2026-04-04 06:10:47,825 - lava_callback - INFO - Starting async job submit for node 69ce085223eb053795d43204 2026-04-04 06:10:47,826 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:10:47,842 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce085223eb053795d43204 HTTP/1.1" 200 None 2026-04-04 06:10:47,843 - lava_callback - DEBUG - Processing node 69ce085223eb053795d43204: baseline-arm64-clabbe 2026-04-04 06:10:47,843 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce085223eb053795d43204 2026-04-04 06:10:47,844 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:10:47,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce085223eb053795d43204 HTTP/1.1" 200 None 2026-04-04 06:10:47,858 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:10:47,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 06:10:47,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:10:47,891 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce085223eb053795d43204 HTTP/1.1" 200 None 2026-04-04 06:10:47,892 - lava_callback - ERROR - Error processing callback for node 69ce085223eb053795d43204: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55978 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43006 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49924 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35516 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:12:07,867 - lava_callback - INFO - Received callback for node 69ce08a223eb053795d4320b INFO: 172.20.0.1:35520 - "POST /node/69ce08a223eb053795d4320b HTTP/1.0" 202 Accepted 2026-04-04 06:12:07,872 - lava_callback - INFO - Starting async job submit for node 69ce08a223eb053795d4320b 2026-04-04 06:12:07,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:12:07,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce08a223eb053795d4320b HTTP/1.1" 200 None 2026-04-04 06:12:07,888 - lava_callback - DEBUG - Processing node 69ce08a223eb053795d4320b: baseline-arm64-clabbe 2026-04-04 06:12:07,889 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce08a223eb053795d4320b 2026-04-04 06:12:07,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:12:07,904 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce08a223eb053795d4320b HTTP/1.1" 200 None 2026-04-04 06:12:07,905 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:12:07,919 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69caa25b4b5e3e0ab6a60b66 HTTP/1.1" 200 None 2026-04-04 06:12:07,921 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:12:07,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce08a223eb053795d4320b HTTP/1.1" 200 None 2026-04-04 06:12:07,953 - lava_callback - ERROR - Error processing callback for node 69ce08a223eb053795d4320b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35550 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60072 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44064 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46638 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46644 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46654 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37610 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40736 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49844 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48426 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35708 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33522 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33482 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55322 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55326 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55330 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55198 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38102 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59382 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39200 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55166 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51208 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51212 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42674 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35136 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37276 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44832 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40670 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47070 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46234 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51468 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59584 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57944 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44320 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55046 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46092 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42418 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36920 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44432 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44438 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57902 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44192 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55110 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42142 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42154 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42160 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45004 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38790 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37500 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49994 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37884 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45280 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39572 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50770 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50774 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43854 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54368 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39336 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52722 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42104 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37870 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58086 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54144 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56026 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48348 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48356 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52250 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48810 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33748 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50712 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53426 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58550 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39122 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52990 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33244 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33252 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58966 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43238 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37288 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37304 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37312 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36752 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56618 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56628 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34840 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37000 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37014 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33454 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44986 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33840 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:44:11,715 - lava_callback - INFO - Received callback for node 69ce102c23eb053795d4325a INFO: 172.20.0.1:33846 - "POST /node/69ce102c23eb053795d4325a HTTP/1.0" 202 Accepted 2026-04-04 06:44:11,720 - lava_callback - INFO - Starting async job submit for node 69ce102c23eb053795d4325a 2026-04-04 06:44:11,721 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:44:11,736 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce102c23eb053795d4325a HTTP/1.1" 200 None 2026-04-04 06:44:11,736 - lava_callback - DEBUG - Processing node 69ce102c23eb053795d4325a: baseline-arm-baylibre 2026-04-04 06:44:11,737 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce102c23eb053795d4325a 2026-04-04 06:44:11,737 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:44:11,751 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce102c23eb053795d4325a HTTP/1.1" 200 None 2026-04-04 06:44:11,752 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:44:11,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 06:44:11,766 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:44:11,788 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce102c23eb053795d4325a HTTP/1.1" 200 None 2026-04-04 06:44:11,789 - lava_callback - ERROR - Error processing callback for node 69ce102c23eb053795d4325a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33848 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33858 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58972 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:45:11,724 - lava_callback - INFO - Received callback for node 69ce106823eb053795d4326b INFO: 172.20.0.1:58980 - "POST /node/69ce106823eb053795d4326b HTTP/1.0" 202 Accepted 2026-04-04 06:45:11,729 - lava_callback - INFO - Starting async job submit for node 69ce106823eb053795d4326b 2026-04-04 06:45:11,730 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:45:11,755 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce106823eb053795d4326b HTTP/1.1" 200 None 2026-04-04 06:45:11,756 - lava_callback - DEBUG - Processing node 69ce106823eb053795d4326b: baseline-arm-baylibre 2026-04-04 06:45:11,756 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce106823eb053795d4326b 2026-04-04 06:45:11,757 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:45:11,770 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce106823eb053795d4326b HTTP/1.1" 200 None 2026-04-04 06:45:11,772 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:45:11,785 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 06:45:11,786 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:45:11,810 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce106823eb053795d4326b HTTP/1.1" 200 None 2026-04-04 06:45:11,811 - lava_callback - ERROR - Error processing callback for node 69ce106823eb053795d4326b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58990 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59002 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39050 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53718 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59658 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52784 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33680 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39798 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56794 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44124 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58040 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55136 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:47:31,966 - lava_callback - INFO - Received callback for node 69ce10f423eb053795d432c1 INFO: 172.20.0.1:41116 - "POST /node/69ce10f423eb053795d432c1 HTTP/1.0" 202 Accepted 2026-04-04 06:47:31,972 - lava_callback - INFO - Starting async job submit for node 69ce10f423eb053795d432c1 2026-04-04 06:47:31,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:47:31,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce10f423eb053795d432c1 HTTP/1.1" 200 None 2026-04-04 06:47:31,993 - lava_callback - DEBUG - Processing node 69ce10f423eb053795d432c1: baseline-arm-baylibre 2026-04-04 06:47:31,993 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce10f423eb053795d432c1 2026-04-04 06:47:31,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:47:32,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce10f423eb053795d432c1 HTTP/1.1" 200 None 2026-04-04 06:47:32,009 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:47:32,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:47:32,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:47:32,052 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce10f423eb053795d432c1 HTTP/1.1" 200 None 2026-04-04 06:47:32,063 - lava_callback - ERROR - Error processing callback for node 69ce10f423eb053795d432c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33360 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52402 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45148 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:48:11,829 - lava_callback - INFO - Received callback for node 69ce111823eb053795d432ca INFO: 172.20.0.1:45160 - "POST /node/69ce111823eb053795d432ca HTTP/1.0" 202 Accepted 2026-04-04 06:48:11,835 - lava_callback - INFO - Starting async job submit for node 69ce111823eb053795d432ca 2026-04-04 06:48:11,836 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:48:11,850 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce111823eb053795d432ca HTTP/1.1" 200 None 2026-04-04 06:48:11,851 - lava_callback - DEBUG - Processing node 69ce111823eb053795d432ca: baseline-arm-baylibre 2026-04-04 06:48:11,852 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce111823eb053795d432ca 2026-04-04 06:48:11,852 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:48:11,865 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce111823eb053795d432ca HTTP/1.1" 200 None 2026-04-04 06:48:11,867 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:48:11,880 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:48:11,882 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:48:11,901 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce111823eb053795d432ca HTTP/1.1" 200 None 2026-04-04 06:48:11,910 - lava_callback - ERROR - Error processing callback for node 69ce111823eb053795d432ca: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45164 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56150 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:48:31,848 - lava_callback - INFO - Received callback for node 69ce113b23eb053795d432d8 INFO: 172.20.0.1:45670 - "POST /node/69ce113b23eb053795d432d8 HTTP/1.0" 202 Accepted 2026-04-04 06:48:31,854 - lava_callback - INFO - Starting async job submit for node 69ce113b23eb053795d432d8 2026-04-04 06:48:31,855 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:48:31,870 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce113b23eb053795d432d8 HTTP/1.1" 200 None 2026-04-04 06:48:31,871 - lava_callback - DEBUG - Processing node 69ce113b23eb053795d432d8: baseline-arm-baylibre 2026-04-04 06:48:31,871 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce113b23eb053795d432d8 2026-04-04 06:48:31,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:48:31,892 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce113b23eb053795d432d8 HTTP/1.1" 200 None 2026-04-04 06:48:31,894 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:48:31,910 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:48:31,911 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:48:31,934 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce113b23eb053795d432d8 HTTP/1.1" 200 None 2026-04-04 06:48:31,935 - lava_callback - ERROR - Error processing callback for node 69ce113b23eb053795d432d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45676 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48710 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55520 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50734 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:49:11,956 - lava_callback - INFO - Received callback for node 69ce115f23eb053795d432e3 INFO: 172.20.0.1:50742 - "POST /node/69ce115f23eb053795d432e3 HTTP/1.0" 202 Accepted 2026-04-04 06:49:11,961 - lava_callback - INFO - Starting async job submit for node 69ce115f23eb053795d432e3 2026-04-04 06:49:11,962 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:49:11,977 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce115f23eb053795d432e3 HTTP/1.1" 200 None 2026-04-04 06:49:11,978 - lava_callback - DEBUG - Processing node 69ce115f23eb053795d432e3: baseline-arm-baylibre 2026-04-04 06:49:11,978 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce115f23eb053795d432e3 2026-04-04 06:49:11,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:49:11,992 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce115f23eb053795d432e3 HTTP/1.1" 200 None 2026-04-04 06:49:11,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:49:12,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:49:12,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:49:12,035 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce115f23eb053795d432e3 HTTP/1.1" 200 None 2026-04-04 06:49:12,043 - lava_callback - ERROR - Error processing callback for node 69ce115f23eb053795d432e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50754 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:50768 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46294 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48664 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:49:51,924 - lava_callback - INFO - Received callback for node 69ce118523eb053795d432ee INFO: 172.20.0.1:35854 - "POST /node/69ce118523eb053795d432ee HTTP/1.0" 202 Accepted 2026-04-04 06:49:51,929 - lava_callback - INFO - Starting async job submit for node 69ce118523eb053795d432ee 2026-04-04 06:49:51,930 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:49:51,945 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce118523eb053795d432ee HTTP/1.1" 200 None 2026-04-04 06:49:51,946 - lava_callback - DEBUG - Processing node 69ce118523eb053795d432ee: baseline-arm-baylibre 2026-04-04 06:49:51,946 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce118523eb053795d432ee 2026-04-04 06:49:51,947 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:49:51,959 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce118523eb053795d432ee HTTP/1.1" 200 None 2026-04-04 06:49:51,961 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:49:51,981 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:49:51,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:49:52,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce118523eb053795d432ee HTTP/1.1" 200 None 2026-04-04 06:49:52,009 - lava_callback - ERROR - Error processing callback for node 69ce118523eb053795d432ee: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35860 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57394 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57402 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57762 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:50:32,010 - lava_callback - INFO - Received callback for node 69ce11aa23eb053795d432f9 INFO: 172.20.0.1:34338 - "POST /node/69ce11aa23eb053795d432f9 HTTP/1.0" 202 Accepted 2026-04-04 06:50:32,016 - lava_callback - INFO - Starting async job submit for node 69ce11aa23eb053795d432f9 2026-04-04 06:50:32,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:50:32,031 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce11aa23eb053795d432f9 HTTP/1.1" 200 None 2026-04-04 06:50:32,032 - lava_callback - DEBUG - Processing node 69ce11aa23eb053795d432f9: baseline-arm-baylibre 2026-04-04 06:50:32,032 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce11aa23eb053795d432f9 2026-04-04 06:50:32,033 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:50:32,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce11aa23eb053795d432f9 HTTP/1.1" 200 None 2026-04-04 06:50:32,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:50:32,060 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:50:32,061 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:50:32,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce11aa23eb053795d432f9 HTTP/1.1" 200 None 2026-04-04 06:50:32,095 - lava_callback - ERROR - Error processing callback for node 69ce11aa23eb053795d432f9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36782 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38074 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38084 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39274 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37076 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37406 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37416 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55594 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:52:29,650 - lava_callback - INFO - Received callback for node 69ce121823eb053795d4331b INFO: 172.20.0.1:55610 - "POST /node/69ce121823eb053795d4331b HTTP/1.0" 202 Accepted 2026-04-04 06:52:29,656 - lava_callback - INFO - Starting async job submit for node 69ce121823eb053795d4331b 2026-04-04 06:52:29,657 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:52:29,672 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce121823eb053795d4331b HTTP/1.1" 200 None 2026-04-04 06:52:29,673 - lava_callback - DEBUG - Processing node 69ce121823eb053795d4331b: baseline-arm-clabbe 2026-04-04 06:52:29,673 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce121823eb053795d4331b 2026-04-04 06:52:29,674 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:52:29,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce121823eb053795d4331b HTTP/1.1" 200 None 2026-04-04 06:52:29,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:52:29,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:52:29,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:52:29,728 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce121823eb053795d4331b HTTP/1.1" 200 None 2026-04-04 06:52:29,733 - lava_callback - ERROR - Error processing callback for node 69ce121823eb053795d4331b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37758 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:52:49,584 - lava_callback - INFO - Received callback for node 69ce123c23eb053795d43327 INFO: 172.20.0.1:37762 - "POST /node/69ce123c23eb053795d43327 HTTP/1.0" 202 Accepted 2026-04-04 06:52:49,590 - lava_callback - INFO - Starting async job submit for node 69ce123c23eb053795d43327 2026-04-04 06:52:49,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:52:49,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce123c23eb053795d43327 HTTP/1.1" 200 None 2026-04-04 06:52:49,607 - lava_callback - DEBUG - Processing node 69ce123c23eb053795d43327: baseline-arm-clabbe 2026-04-04 06:52:49,607 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce123c23eb053795d43327 2026-04-04 06:52:49,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:52:49,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce123c23eb053795d43327 HTTP/1.1" 200 None 2026-04-04 06:52:49,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:52:49,643 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:52:49,644 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:52:49,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce123c23eb053795d43327 HTTP/1.1" 200 None 2026-04-04 06:52:49,669 - lava_callback - ERROR - Error processing callback for node 69ce123c23eb053795d43327: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44696 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36302 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36316 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36324 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49716 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:53:29,643 - lava_callback - INFO - Received callback for node 69ce126123eb053795d43331 INFO: 172.20.0.1:49720 - "POST /node/69ce126123eb053795d43331 HTTP/1.0" 202 Accepted 2026-04-04 06:53:29,649 - lava_callback - INFO - Starting async job submit for node 69ce126123eb053795d43331 2026-04-04 06:53:29,650 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:53:29,664 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce126123eb053795d43331 HTTP/1.1" 200 None 2026-04-04 06:53:29,664 - lava_callback - DEBUG - Processing node 69ce126123eb053795d43331: baseline-arm-clabbe 2026-04-04 06:53:29,665 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce126123eb053795d43331 2026-04-04 06:53:29,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:53:29,679 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce126123eb053795d43331 HTTP/1.1" 200 None 2026-04-04 06:53:29,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:53:29,696 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:53:29,698 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:53:29,718 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce126123eb053795d43331 HTTP/1.1" 200 None 2026-04-04 06:53:29,725 - lava_callback - ERROR - Error processing callback for node 69ce126123eb053795d43331: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45652 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38380 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45818 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:54:09,599 - lava_callback - INFO - Received callback for node 69ce128623eb053795d43342 INFO: 172.20.0.1:45824 - "POST /node/69ce128623eb053795d43342 HTTP/1.0" 202 Accepted 2026-04-04 06:54:09,606 - lava_callback - INFO - Starting async job submit for node 69ce128623eb053795d43342 2026-04-04 06:54:09,608 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:54:09,622 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce128623eb053795d43342 HTTP/1.1" 200 None 2026-04-04 06:54:09,623 - lava_callback - DEBUG - Processing node 69ce128623eb053795d43342: baseline-arm-clabbe 2026-04-04 06:54:09,623 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce128623eb053795d43342 2026-04-04 06:54:09,624 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:54:09,637 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce128623eb053795d43342 HTTP/1.1" 200 None 2026-04-04 06:54:09,639 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:54:09,661 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:54:09,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:54:09,685 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce128623eb053795d43342 HTTP/1.1" 200 None 2026-04-04 06:54:09,690 - lava_callback - ERROR - Error processing callback for node 69ce128623eb053795d43342: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37972 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55746 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:54:49,664 - lava_callback - INFO - Received callback for node 69ce12aa23eb053795d4334d INFO: 172.20.0.1:55750 - "POST /node/69ce12aa23eb053795d4334d HTTP/1.0" 202 Accepted 2026-04-04 06:54:49,672 - lava_callback - INFO - Starting async job submit for node 69ce12aa23eb053795d4334d 2026-04-04 06:54:49,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:54:49,688 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce12aa23eb053795d4334d HTTP/1.1" 200 None 2026-04-04 06:54:49,689 - lava_callback - DEBUG - Processing node 69ce12aa23eb053795d4334d: baseline-arm-clabbe 2026-04-04 06:54:49,689 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce12aa23eb053795d4334d 2026-04-04 06:54:49,690 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:54:49,709 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce12aa23eb053795d4334d HTTP/1.1" 200 None 2026-04-04 06:54:49,710 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:54:49,724 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:54:49,725 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:54:49,746 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce12aa23eb053795d4334d HTTP/1.1" 200 None 2026-04-04 06:54:49,753 - lava_callback - ERROR - Error processing callback for node 69ce12aa23eb053795d4334d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60752 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60756 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50256 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:55:29,658 - lava_callback - INFO - Received callback for node 69ce12d023eb053795d43358 INFO: 172.20.0.1:50266 - "POST /node/69ce12d023eb053795d43358 HTTP/1.0" 202 Accepted 2026-04-04 06:55:29,663 - lava_callback - INFO - Starting async job submit for node 69ce12d023eb053795d43358 2026-04-04 06:55:29,664 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:55:29,678 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce12d023eb053795d43358 HTTP/1.1" 200 None 2026-04-04 06:55:29,679 - lava_callback - DEBUG - Processing node 69ce12d023eb053795d43358: baseline-arm-clabbe 2026-04-04 06:55:29,679 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce12d023eb053795d43358 2026-04-04 06:55:29,680 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:55:29,693 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce12d023eb053795d43358 HTTP/1.1" 200 None 2026-04-04 06:55:29,694 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:55:29,708 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5b723eb053795d41eb7 HTTP/1.1" 200 None 2026-04-04 06:55:29,709 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:55:29,730 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce12d023eb053795d43358 HTTP/1.1" 200 None 2026-04-04 06:55:29,732 - lava_callback - ERROR - Error processing callback for node 69ce12d023eb053795d43358: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49888 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35876 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:55:52,416 - lava_callback - INFO - Received callback for node 69ce12f123eb053795d43363 INFO: 172.20.0.1:38456 - "POST /node/69ce12f123eb053795d43363 HTTP/1.0" 202 Accepted 2026-04-04 06:55:52,425 - lava_callback - INFO - Starting async job submit for node 69ce12f123eb053795d43363 2026-04-04 06:55:52,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:55:52,450 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce12f123eb053795d43363 HTTP/1.1" 200 None 2026-04-04 06:55:52,451 - lava_callback - DEBUG - Processing node 69ce12f123eb053795d43363: baseline-arm-baylibre 2026-04-04 06:55:52,451 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce12f123eb053795d43363 2026-04-04 06:55:52,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:55:52,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce12f123eb053795d43363 HTTP/1.1" 200 None 2026-04-04 06:55:52,475 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:55:52,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 06:55:52,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:55:52,517 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce12f123eb053795d43363 HTTP/1.1" 200 None 2026-04-04 06:55:52,523 - lava_callback - ERROR - Error processing callback for node 69ce12f123eb053795d43363: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38472 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47830 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49052 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45676 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:56:32,332 - lava_callback - INFO - Received callback for node 69ce131523eb053795d4336e INFO: 172.20.0.1:46246 - "POST /node/69ce131523eb053795d4336e HTTP/1.0" 202 Accepted 2026-04-04 06:56:32,338 - lava_callback - INFO - Starting async job submit for node 69ce131523eb053795d4336e 2026-04-04 06:56:32,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:56:32,361 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce131523eb053795d4336e HTTP/1.1" 200 None 2026-04-04 06:56:32,362 - lava_callback - DEBUG - Processing node 69ce131523eb053795d4336e: baseline-arm-baylibre 2026-04-04 06:56:32,362 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce131523eb053795d4336e 2026-04-04 06:56:32,363 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:56:32,376 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce131523eb053795d4336e HTTP/1.1" 200 None 2026-04-04 06:56:32,377 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:56:32,390 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 06:56:32,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:56:32,411 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce131523eb053795d4336e HTTP/1.1" 200 None 2026-04-04 06:56:32,412 - lava_callback - ERROR - Error processing callback for node 69ce131523eb053795d4336e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46560 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49706 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56734 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:57:12,358 - lava_callback - INFO - Received callback for node 69ce133823eb053795d4337d INFO: 172.20.0.1:56748 - "POST /node/69ce133823eb053795d4337d HTTP/1.0" 202 Accepted 2026-04-04 06:57:12,363 - lava_callback - INFO - Starting async job submit for node 69ce133823eb053795d4337d 2026-04-04 06:57:12,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:57:12,383 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce133823eb053795d4337d HTTP/1.1" 200 None 2026-04-04 06:57:12,384 - lava_callback - DEBUG - Processing node 69ce133823eb053795d4337d: baseline-arm-baylibre 2026-04-04 06:57:12,384 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce133823eb053795d4337d 2026-04-04 06:57:12,385 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:57:12,409 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce133823eb053795d4337d HTTP/1.1" 200 None 2026-04-04 06:57:12,410 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:57:12,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 06:57:12,426 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:57:12,447 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce133823eb053795d4337d HTTP/1.1" 200 None 2026-04-04 06:57:12,449 - lava_callback - ERROR - Error processing callback for node 69ce133823eb053795d4337d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37256 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34848 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:57:52,385 - lava_callback - INFO - Received callback for node 69ce135d23eb053795d43387 INFO: 172.20.0.1:51374 - "POST /node/69ce135d23eb053795d43387 HTTP/1.0" 202 Accepted 2026-04-04 06:57:52,391 - lava_callback - INFO - Starting async job submit for node 69ce135d23eb053795d43387 2026-04-04 06:57:52,392 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:57:52,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce135d23eb053795d43387 HTTP/1.1" 200 None 2026-04-04 06:57:52,416 - lava_callback - DEBUG - Processing node 69ce135d23eb053795d43387: baseline-arm-baylibre 2026-04-04 06:57:52,416 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce135d23eb053795d43387 2026-04-04 06:57:52,417 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:57:52,430 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce135d23eb053795d43387 HTTP/1.1" 200 None 2026-04-04 06:57:52,432 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:57:52,445 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 06:57:52,447 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:57:52,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce135d23eb053795d43387 HTTP/1.1" 200 None 2026-04-04 06:57:52,470 - lava_callback - ERROR - Error processing callback for node 69ce135d23eb053795d43387: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49484 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:58:12,408 - lava_callback - INFO - Received callback for node 69ce138123eb053795d43394 INFO: 172.20.0.1:49488 - "POST /node/69ce138123eb053795d43394 HTTP/1.0" 202 Accepted 2026-04-04 06:58:12,417 - lava_callback - INFO - Starting async job submit for node 69ce138123eb053795d43394 2026-04-04 06:58:12,418 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:58:12,437 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce138123eb053795d43394 HTTP/1.1" 200 None 2026-04-04 06:58:12,438 - lava_callback - DEBUG - Processing node 69ce138123eb053795d43394: baseline-arm-baylibre 2026-04-04 06:58:12,438 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce138123eb053795d43394 2026-04-04 06:58:12,439 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:58:12,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce138123eb053795d43394 HTTP/1.1" 200 None 2026-04-04 06:58:12,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:58:12,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 06:58:12,477 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:58:12,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce138123eb053795d43394 HTTP/1.1" 200 None 2026-04-04 06:58:12,501 - lava_callback - ERROR - Error processing callback for node 69ce138123eb053795d43394: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:49494 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49500 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55912 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54250 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:58:52,479 - lava_callback - INFO - Received callback for node 69ce13a523eb053795d4339f INFO: 172.20.0.1:48032 - "POST /node/69ce13a523eb053795d4339f HTTP/1.0" 202 Accepted 2026-04-04 06:58:52,484 - lava_callback - INFO - Starting async job submit for node 69ce13a523eb053795d4339f 2026-04-04 06:58:52,485 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:58:52,498 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce13a523eb053795d4339f HTTP/1.1" 200 None 2026-04-04 06:58:52,499 - lava_callback - DEBUG - Processing node 69ce13a523eb053795d4339f: baseline-arm-baylibre 2026-04-04 06:58:52,499 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce13a523eb053795d4339f 2026-04-04 06:58:52,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:58:52,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce13a523eb053795d4339f HTTP/1.1" 200 None 2026-04-04 06:58:52,513 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:58:52,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 06:58:52,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:58:52,555 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce13a523eb053795d4339f HTTP/1.1" 200 None 2026-04-04 06:58:52,557 - lava_callback - ERROR - Error processing callback for node 69ce13a523eb053795d4339f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60702 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38552 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38564 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40510 - "GET / HTTP/1.0" 200 OK 2026-04-04 06:59:29,902 - lava_callback - INFO - Received callback for node 69ce13ca23eb053795d433ac INFO: 172.20.0.1:48452 - "POST /node/69ce13ca23eb053795d433ac HTTP/1.0" 202 Accepted 2026-04-04 06:59:29,907 - lava_callback - INFO - Starting async job submit for node 69ce13ca23eb053795d433ac 2026-04-04 06:59:29,908 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:59:29,922 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce13ca23eb053795d433ac HTTP/1.1" 200 None 2026-04-04 06:59:29,923 - lava_callback - DEBUG - Processing node 69ce13ca23eb053795d433ac: baseline-arm-clabbe 2026-04-04 06:59:29,923 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce13ca23eb053795d433ac 2026-04-04 06:59:29,924 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:59:29,947 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce13ca23eb053795d433ac HTTP/1.1" 200 None 2026-04-04 06:59:29,949 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:59:29,962 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 06:59:29,964 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 06:59:29,984 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce13ca23eb053795d433ac HTTP/1.1" 200 None 2026-04-04 06:59:29,986 - lava_callback - ERROR - Error processing callback for node 69ce13ca23eb053795d433ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58548 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44150 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:00:10,126 - lava_callback - INFO - Received callback for node 69ce13ef23eb053795d433b9 INFO: 172.20.0.1:40064 - "POST /node/69ce13ef23eb053795d433b9 HTTP/1.0" 202 Accepted 2026-04-04 07:00:10,131 - lava_callback - INFO - Starting async job submit for node 69ce13ef23eb053795d433b9 2026-04-04 07:00:10,132 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:00:10,146 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce13ef23eb053795d433b9 HTTP/1.1" 200 None 2026-04-04 07:00:10,147 - lava_callback - DEBUG - Processing node 69ce13ef23eb053795d433b9: baseline-arm-clabbe 2026-04-04 07:00:10,147 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce13ef23eb053795d433b9 2026-04-04 07:00:10,148 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:00:10,168 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce13ef23eb053795d433b9 HTTP/1.1" 200 None 2026-04-04 07:00:10,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:00:10,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:00:10,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:00:10,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce13ef23eb053795d433b9 HTTP/1.1" 200 None 2026-04-04 07:00:10,211 - lava_callback - ERROR - Error processing callback for node 69ce13ef23eb053795d433b9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40072 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46038 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51090 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:00:49,968 - lava_callback - INFO - Received callback for node 69ce141423eb053795d433c2 INFO: 172.20.0.1:50640 - "POST /node/69ce141423eb053795d433c2 HTTP/1.0" 202 Accepted 2026-04-04 07:00:49,977 - lava_callback - INFO - Starting async job submit for node 69ce141423eb053795d433c2 2026-04-04 07:00:49,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:00:49,993 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce141423eb053795d433c2 HTTP/1.1" 200 None 2026-04-04 07:00:49,994 - lava_callback - DEBUG - Processing node 69ce141423eb053795d433c2: baseline-arm-clabbe 2026-04-04 07:00:49,995 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce141423eb053795d433c2 2026-04-04 07:00:49,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:00:50,010 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce141423eb053795d433c2 HTTP/1.1" 200 None 2026-04-04 07:00:50,012 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:00:50,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:00:50,030 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:00:50,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce141423eb053795d433c2 HTTP/1.1" 200 None 2026-04-04 07:00:50,057 - lava_callback - ERROR - Error processing callback for node 69ce141423eb053795d433c2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47188 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40754 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44402 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:01:29,947 - lava_callback - INFO - Received callback for node 69ce143a23eb053795d433cc INFO: 172.20.0.1:35000 - "POST /node/69ce143a23eb053795d433cc HTTP/1.0" 202 Accepted 2026-04-04 07:01:29,952 - lava_callback - INFO - Starting async job submit for node 69ce143a23eb053795d433cc 2026-04-04 07:01:29,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:01:29,972 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce143a23eb053795d433cc HTTP/1.1" 200 None 2026-04-04 07:01:29,973 - lava_callback - DEBUG - Processing node 69ce143a23eb053795d433cc: baseline-arm-clabbe 2026-04-04 07:01:29,973 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce143a23eb053795d433cc 2026-04-04 07:01:29,974 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:01:29,987 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce143a23eb053795d433cc HTTP/1.1" 200 None 2026-04-04 07:01:29,988 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:01:30,001 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:01:30,003 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:01:30,028 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce143a23eb053795d433cc HTTP/1.1" 200 None 2026-04-04 07:01:30,029 - lava_callback - ERROR - Error processing callback for node 69ce143a23eb053795d433cc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35014 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36044 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58036 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:02:09,979 - lava_callback - INFO - Received callback for node 69ce145e23eb053795d433d6 INFO: 172.20.0.1:48522 - "POST /node/69ce145e23eb053795d433d6 HTTP/1.0" 202 Accepted 2026-04-04 07:02:09,985 - lava_callback - INFO - Starting async job submit for node 69ce145e23eb053795d433d6 2026-04-04 07:02:09,986 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:02:10,000 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce145e23eb053795d433d6 HTTP/1.1" 200 None 2026-04-04 07:02:10,001 - lava_callback - DEBUG - Processing node 69ce145e23eb053795d433d6: baseline-arm-clabbe 2026-04-04 07:02:10,001 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce145e23eb053795d433d6 2026-04-04 07:02:10,001 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:02:10,014 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce145e23eb053795d433d6 HTTP/1.1" 200 None 2026-04-04 07:02:10,016 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:02:10,029 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:02:10,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:02:10,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce145e23eb053795d433d6 HTTP/1.1" 200 None 2026-04-04 07:02:10,059 - lava_callback - ERROR - Error processing callback for node 69ce145e23eb053795d433d6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48542 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59270 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50534 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:02:49,974 - lava_callback - INFO - Received callback for node 69ce148423eb053795d433e2 INFO: 172.20.0.1:34482 - "POST /node/69ce148423eb053795d433e2 HTTP/1.0" 202 Accepted 2026-04-04 07:02:49,981 - lava_callback - INFO - Starting async job submit for node 69ce148423eb053795d433e2 2026-04-04 07:02:49,982 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:02:49,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce148423eb053795d433e2 HTTP/1.1" 200 None 2026-04-04 07:02:49,997 - lava_callback - DEBUG - Processing node 69ce148423eb053795d433e2: baseline-arm-clabbe 2026-04-04 07:02:49,998 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce148423eb053795d433e2 2026-04-04 07:02:49,998 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:02:50,017 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce148423eb053795d433e2 HTTP/1.1" 200 None 2026-04-04 07:02:50,018 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:02:50,037 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:02:50,039 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:02:50,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce148423eb053795d433e2 HTTP/1.1" 200 None 2026-04-04 07:02:50,074 - lava_callback - ERROR - Error processing callback for node 69ce148423eb053795d433e2: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46462 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:03:09,985 - lava_callback - INFO - Received callback for node 69ce14a823eb053795d433ea INFO: 172.20.0.1:42058 - "POST /node/69ce14a823eb053795d433ea HTTP/1.0" 202 Accepted 2026-04-04 07:03:09,994 - lava_callback - INFO - Starting async job submit for node 69ce14a823eb053795d433ea 2026-04-04 07:03:09,996 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:03:10,051 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce14a823eb053795d433ea HTTP/1.1" 200 None 2026-04-04 07:03:10,052 - lava_callback - DEBUG - Processing node 69ce14a823eb053795d433ea: baseline-arm-clabbe 2026-04-04 07:03:10,053 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce14a823eb053795d433ea 2026-04-04 07:03:10,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:03:10,091 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce14a823eb053795d433ea HTTP/1.1" 200 None 2026-04-04 07:03:10,094 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:03:10,138 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:03:10,141 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:03:10,171 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce14a823eb053795d433ea HTTP/1.1" 200 None 2026-04-04 07:03:10,173 - lava_callback - ERROR - Error processing callback for node 69ce14a823eb053795d433ea: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42068 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51784 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53192 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:03:50,016 - lava_callback - INFO - Received callback for node 69ce14cd23eb053795d433f4 INFO: 172.20.0.1:33796 - "POST /node/69ce14cd23eb053795d433f4 HTTP/1.0" 202 Accepted 2026-04-04 07:03:50,021 - lava_callback - INFO - Starting async job submit for node 69ce14cd23eb053795d433f4 2026-04-04 07:03:50,022 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:03:50,036 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce14cd23eb053795d433f4 HTTP/1.1" 200 None 2026-04-04 07:03:50,037 - lava_callback - DEBUG - Processing node 69ce14cd23eb053795d433f4: baseline-arm-clabbe 2026-04-04 07:03:50,037 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce14cd23eb053795d433f4 2026-04-04 07:03:50,038 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:03:50,057 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce14cd23eb053795d433f4 HTTP/1.1" 200 None 2026-04-04 07:03:50,059 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:03:50,073 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:03:50,074 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:03:50,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce14cd23eb053795d433f4 HTTP/1.1" 200 None 2026-04-04 07:03:50,096 - lava_callback - ERROR - Error processing callback for node 69ce14cd23eb053795d433f4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:33800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48088 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55482 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50370 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:04:30,089 - lava_callback - INFO - Received callback for node 69ce14f123eb053795d43404 INFO: 172.20.0.1:36492 - "POST /node/69ce14f123eb053795d43404 HTTP/1.0" 202 Accepted 2026-04-04 07:04:30,094 - lava_callback - INFO - Starting async job submit for node 69ce14f123eb053795d43404 2026-04-04 07:04:30,095 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:04:30,113 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce14f123eb053795d43404 HTTP/1.1" 200 None 2026-04-04 07:04:30,114 - lava_callback - DEBUG - Processing node 69ce14f123eb053795d43404: baseline-arm-clabbe 2026-04-04 07:04:30,114 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce14f123eb053795d43404 2026-04-04 07:04:30,114 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:04:30,131 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce14f123eb053795d43404 HTTP/1.1" 200 None 2026-04-04 07:04:30,133 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:04:30,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:04:30,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:04:30,170 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce14f123eb053795d43404 HTTP/1.1" 200 None 2026-04-04 07:04:30,171 - lava_callback - ERROR - Error processing callback for node 69ce14f123eb053795d43404: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36896 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39000 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:05:10,029 - lava_callback - INFO - Received callback for node 69ce151523eb053795d43410 INFO: 172.20.0.1:52910 - "POST /node/69ce151523eb053795d43410 HTTP/1.0" 202 Accepted 2026-04-04 07:05:10,034 - lava_callback - INFO - Starting async job submit for node 69ce151523eb053795d43410 2026-04-04 07:05:10,035 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:05:10,053 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce151523eb053795d43410 HTTP/1.1" 200 None 2026-04-04 07:05:10,054 - lava_callback - DEBUG - Processing node 69ce151523eb053795d43410: baseline-arm-clabbe 2026-04-04 07:05:10,054 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce151523eb053795d43410 2026-04-04 07:05:10,054 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:05:10,068 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce151523eb053795d43410 HTTP/1.1" 200 None 2026-04-04 07:05:10,070 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:05:10,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:05:10,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:05:10,104 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce151523eb053795d43410 HTTP/1.1" 200 None 2026-04-04 07:05:10,106 - lava_callback - ERROR - Error processing callback for node 69ce151523eb053795d43410: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52928 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52930 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57786 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47160 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:05:50,136 - lava_callback - INFO - Received callback for node 69ce153923eb053795d4341e INFO: 172.20.0.1:52712 - "POST /node/69ce153923eb053795d4341e HTTP/1.0" 202 Accepted 2026-04-04 07:05:50,141 - lava_callback - INFO - Starting async job submit for node 69ce153923eb053795d4341e 2026-04-04 07:05:50,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:05:50,172 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce153923eb053795d4341e HTTP/1.1" 200 None 2026-04-04 07:05:50,173 - lava_callback - DEBUG - Processing node 69ce153923eb053795d4341e: baseline-arm-clabbe 2026-04-04 07:05:50,173 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce153923eb053795d4341e 2026-04-04 07:05:50,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:05:50,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce153923eb053795d4341e HTTP/1.1" 200 None 2026-04-04 07:05:50,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:05:50,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:05:50,205 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:05:50,229 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce153923eb053795d4341e HTTP/1.1" 200 None 2026-04-04 07:05:50,231 - lava_callback - ERROR - Error processing callback for node 69ce153923eb053795d4341e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53558 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:06:10,089 - lava_callback - INFO - Received callback for node 69ce155e23eb053795d43429 INFO: 172.20.0.1:40654 - "POST /node/69ce155e23eb053795d43429 HTTP/1.0" 202 Accepted 2026-04-04 07:06:10,095 - lava_callback - INFO - Starting async job submit for node 69ce155e23eb053795d43429 2026-04-04 07:06:10,096 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:06:10,109 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce155e23eb053795d43429 HTTP/1.1" 200 None 2026-04-04 07:06:10,110 - lava_callback - DEBUG - Processing node 69ce155e23eb053795d43429: baseline-arm-clabbe 2026-04-04 07:06:10,111 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce155e23eb053795d43429 2026-04-04 07:06:10,111 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:06:10,129 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce155e23eb053795d43429 HTTP/1.1" 200 None 2026-04-04 07:06:10,131 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:06:10,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:06:10,146 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:06:10,178 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce155e23eb053795d43429 HTTP/1.1" 200 None 2026-04-04 07:06:10,179 - lava_callback - ERROR - Error processing callback for node 69ce155e23eb053795d43429: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40680 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57878 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32972 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:06:50,168 - lava_callback - INFO - Received callback for node 69ce158223eb053795d43435 INFO: 172.20.0.1:48478 - "POST /node/69ce158223eb053795d43435 HTTP/1.0" 202 Accepted 2026-04-04 07:06:50,173 - lava_callback - INFO - Starting async job submit for node 69ce158223eb053795d43435 2026-04-04 07:06:50,174 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:06:50,200 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce158223eb053795d43435 HTTP/1.1" 200 None 2026-04-04 07:06:50,201 - lava_callback - DEBUG - Processing node 69ce158223eb053795d43435: baseline-arm-clabbe 2026-04-04 07:06:50,201 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce158223eb053795d43435 2026-04-04 07:06:50,202 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:06:50,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce158223eb053795d43435 HTTP/1.1" 200 None 2026-04-04 07:06:50,218 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:06:50,231 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:06:50,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:06:50,253 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce158223eb053795d43435 HTTP/1.1" 200 None 2026-04-04 07:06:50,259 - lava_callback - ERROR - Error processing callback for node 69ce158223eb053795d43435: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:48490 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45340 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45358 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55686 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:07:30,165 - lava_callback - INFO - Received callback for node 69ce15a623eb053795d43446 INFO: 172.20.0.1:50498 - "POST /node/69ce15a623eb053795d43446 HTTP/1.0" 202 Accepted 2026-04-04 07:07:30,170 - lava_callback - INFO - Starting async job submit for node 69ce15a623eb053795d43446 2026-04-04 07:07:30,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:07:30,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce15a623eb053795d43446 HTTP/1.1" 200 None 2026-04-04 07:07:30,189 - lava_callback - DEBUG - Processing node 69ce15a623eb053795d43446: baseline-arm-clabbe 2026-04-04 07:07:30,190 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce15a623eb053795d43446 2026-04-04 07:07:30,190 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:07:30,203 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce15a623eb053795d43446 HTTP/1.1" 200 None 2026-04-04 07:07:30,204 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:07:30,217 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:07:30,219 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:07:30,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce15a623eb053795d43446 HTTP/1.1" 200 None 2026-04-04 07:07:30,241 - lava_callback - ERROR - Error processing callback for node 69ce15a623eb053795d43446: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50510 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32892 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36180 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:08:10,247 - lava_callback - INFO - Received callback for node 69ce15cb23eb053795d4345d INFO: 172.20.0.1:55692 - "POST /node/69ce15cb23eb053795d4345d HTTP/1.0" 202 Accepted 2026-04-04 07:08:10,252 - lava_callback - INFO - Starting async job submit for node 69ce15cb23eb053795d4345d 2026-04-04 07:08:10,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:08:10,267 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce15cb23eb053795d4345d HTTP/1.1" 200 None 2026-04-04 07:08:10,268 - lava_callback - DEBUG - Processing node 69ce15cb23eb053795d4345d: baseline-arm-clabbe 2026-04-04 07:08:10,268 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce15cb23eb053795d4345d 2026-04-04 07:08:10,269 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:08:10,282 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce15cb23eb053795d4345d HTTP/1.1" 200 None 2026-04-04 07:08:10,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:08:10,297 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f72f41e4fb6a3782f3e HTTP/1.1" 200 None 2026-04-04 07:08:10,298 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:08:10,318 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce15cb23eb053795d4345d HTTP/1.1" 200 None 2026-04-04 07:08:10,323 - lava_callback - ERROR - Error processing callback for node 69ce15cb23eb053795d4345d: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:55704 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55718 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44076 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:08:33,359 - lava_callback - INFO - Received callback for node 69ce15eb23eb053795d4346c INFO: 172.20.0.1:52380 - "POST /node/69ce15eb23eb053795d4346c HTTP/1.0" 202 Accepted 2026-04-04 07:08:33,364 - lava_callback - INFO - Starting async job submit for node 69ce15eb23eb053795d4346c 2026-04-04 07:08:33,365 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:08:33,380 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce15eb23eb053795d4346c HTTP/1.1" 200 None 2026-04-04 07:08:33,381 - lava_callback - DEBUG - Processing node 69ce15eb23eb053795d4346c: baseline-arm-baylibre 2026-04-04 07:08:33,381 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce15eb23eb053795d4346c 2026-04-04 07:08:33,382 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:08:33,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce15eb23eb053795d4346c HTTP/1.1" 200 None 2026-04-04 07:08:33,396 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:08:33,413 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:08:33,415 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:08:33,436 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce15eb23eb053795d4346c HTTP/1.1" 200 None 2026-04-04 07:08:33,437 - lava_callback - ERROR - Error processing callback for node 69ce15eb23eb053795d4346c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34588 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59290 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:09:12,967 - lava_callback - INFO - Received callback for node 69ce160f23eb053795d43490 INFO: 172.20.0.1:59306 - "POST /node/69ce160f23eb053795d43490 HTTP/1.0" 202 Accepted 2026-04-04 07:09:12,981 - lava_callback - INFO - Starting async job submit for node 69ce160f23eb053795d43490 2026-04-04 07:09:12,983 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:09:13,034 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce160f23eb053795d43490 HTTP/1.1" 200 None 2026-04-04 07:09:13,035 - lava_callback - DEBUG - Processing node 69ce160f23eb053795d43490: baseline-arm-baylibre 2026-04-04 07:09:13,036 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce160f23eb053795d43490 2026-04-04 07:09:13,037 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:09:13,055 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce160f23eb053795d43490 HTTP/1.1" 200 None 2026-04-04 07:09:13,056 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:09:13,070 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:09:13,072 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:09:13,093 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce160f23eb053795d43490 HTTP/1.1" 200 None 2026-04-04 07:09:13,095 - lava_callback - ERROR - Error processing callback for node 69ce160f23eb053795d43490: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59314 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56940 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:09:53,061 - lava_callback - INFO - Received callback for node 69ce163323eb053795d434a9 INFO: 172.20.0.1:51942 - "POST /node/69ce163323eb053795d434a9 HTTP/1.0" 202 Accepted 2026-04-04 07:09:53,066 - lava_callback - INFO - Starting async job submit for node 69ce163323eb053795d434a9 2026-04-04 07:09:53,067 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:09:53,081 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce163323eb053795d434a9 HTTP/1.1" 200 None 2026-04-04 07:09:53,082 - lava_callback - DEBUG - Processing node 69ce163323eb053795d434a9: baseline-arm-baylibre 2026-04-04 07:09:53,082 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce163323eb053795d434a9 2026-04-04 07:09:53,083 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:09:53,097 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce163323eb053795d434a9 HTTP/1.1" 200 None 2026-04-04 07:09:53,098 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:09:53,112 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:09:53,113 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:09:53,137 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce163323eb053795d434a9 HTTP/1.1" 200 None 2026-04-04 07:09:53,138 - lava_callback - ERROR - Error processing callback for node 69ce163323eb053795d434a9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60702 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60728 - "GET /node/694a53eb83c67c43d1ed94a7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:60744 - "GET /node/69281220b1b5e06bf6c05129%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54994 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:10:33,102 - lava_callback - INFO - Received callback for node 69ce165723eb053795d434b7 INFO: 172.20.0.1:32934 - "POST /node/69ce165723eb053795d434b7 HTTP/1.0" 202 Accepted 2026-04-04 07:10:33,108 - lava_callback - INFO - Starting async job submit for node 69ce165723eb053795d434b7 2026-04-04 07:10:33,109 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:10:33,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce165723eb053795d434b7 HTTP/1.1" 200 None 2026-04-04 07:10:33,124 - lava_callback - DEBUG - Processing node 69ce165723eb053795d434b7: baseline-arm-baylibre 2026-04-04 07:10:33,124 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce165723eb053795d434b7 2026-04-04 07:10:33,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:10:33,141 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce165723eb053795d434b7 HTTP/1.1" 200 None 2026-04-04 07:10:33,142 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:10:33,155 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:10:33,157 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:10:33,188 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce165723eb053795d434b7 HTTP/1.1" 200 None 2026-04-04 07:10:33,190 - lava_callback - ERROR - Error processing callback for node 69ce165723eb053795d434b7: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59266 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:10:53,128 - lava_callback - INFO - Received callback for node 69ce167923eb053795d434c4 INFO: 172.20.0.1:40320 - "POST /node/69ce167923eb053795d434c4 HTTP/1.0" 202 Accepted 2026-04-04 07:10:53,134 - lava_callback - INFO - Starting async job submit for node 69ce167923eb053795d434c4 2026-04-04 07:10:53,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:10:53,148 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce167923eb053795d434c4 HTTP/1.1" 200 None 2026-04-04 07:10:53,149 - lava_callback - DEBUG - Processing node 69ce167923eb053795d434c4: baseline-arm-baylibre 2026-04-04 07:10:53,149 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce167923eb053795d434c4 2026-04-04 07:10:53,150 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:10:53,163 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce167923eb053795d434c4 HTTP/1.1" 200 None 2026-04-04 07:10:53,165 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:10:53,179 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:10:53,180 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:10:53,209 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce167923eb053795d434c4 HTTP/1.1" 200 None 2026-04-04 07:10:53,210 - lava_callback - ERROR - Error processing callback for node 69ce167923eb053795d434c4: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:40328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41732 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57852 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:57856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46418 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:11:33,254 - lava_callback - INFO - Received callback for node 69ce169d23eb053795d434d3 INFO: 172.20.0.1:60478 - "POST /node/69ce169d23eb053795d434d3 HTTP/1.0" 202 Accepted 2026-04-04 07:11:33,259 - lava_callback - INFO - Starting async job submit for node 69ce169d23eb053795d434d3 2026-04-04 07:11:33,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:11:33,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce169d23eb053795d434d3 HTTP/1.1" 200 None 2026-04-04 07:11:33,274 - lava_callback - DEBUG - Processing node 69ce169d23eb053795d434d3: baseline-arm-baylibre 2026-04-04 07:11:33,274 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce169d23eb053795d434d3 2026-04-04 07:11:33,275 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:11:33,288 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce169d23eb053795d434d3 HTTP/1.1" 200 None 2026-04-04 07:11:33,290 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:11:33,303 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:11:33,304 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:11:33,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce169d23eb053795d434d3 HTTP/1.1" 200 None 2026-04-04 07:11:33,341 - lava_callback - ERROR - Error processing callback for node 69ce169d23eb053795d434d3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:60484 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60884 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:12:10,436 - lava_callback - INFO - Received callback for node 69ce16c523eb053795d434dc INFO: 172.20.0.1:39320 - "POST /node/69ce16c523eb053795d434dc HTTP/1.0" 202 Accepted 2026-04-04 07:12:10,442 - lava_callback - INFO - Starting async job submit for node 69ce16c523eb053795d434dc 2026-04-04 07:12:10,443 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:12:10,459 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce16c523eb053795d434dc HTTP/1.1" 200 None 2026-04-04 07:12:10,461 - lava_callback - DEBUG - Processing node 69ce16c523eb053795d434dc: baseline-arm-clabbe 2026-04-04 07:12:10,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce16c523eb053795d434dc 2026-04-04 07:12:10,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:12:10,476 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce16c523eb053795d434dc HTTP/1.1" 200 None 2026-04-04 07:12:10,478 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:12:10,490 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:12:10,492 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:12:10,512 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce16c523eb053795d434dc HTTP/1.1" 200 None 2026-04-04 07:12:10,515 - lava_callback - ERROR - Error processing callback for node 69ce16c523eb053795d434dc: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39326 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39342 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39354 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56838 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:12:53,228 - lava_callback - INFO - Received callback for node 69ce16e823eb053795d434e9 INFO: 172.20.0.1:52112 - "POST /node/69ce16e823eb053795d434e9 HTTP/1.0" 202 Accepted 2026-04-04 07:12:53,235 - lava_callback - INFO - Starting async job submit for node 69ce16e823eb053795d434e9 2026-04-04 07:12:53,236 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:12:53,258 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce16e823eb053795d434e9 HTTP/1.1" 200 None 2026-04-04 07:12:53,259 - lava_callback - DEBUG - Processing node 69ce16e823eb053795d434e9: baseline-arm-baylibre 2026-04-04 07:12:53,259 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce16e823eb053795d434e9 2026-04-04 07:12:53,260 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:12:53,273 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce16e823eb053795d434e9 HTTP/1.1" 200 None 2026-04-04 07:12:53,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:12:53,287 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:12:53,289 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:12:53,309 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce16e823eb053795d434e9 HTTP/1.1" 200 None 2026-04-04 07:12:53,311 - lava_callback - ERROR - Error processing callback for node 69ce16e823eb053795d434e9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:52114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52352 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47026 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47030 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41926 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:13:33,227 - lava_callback - INFO - Received callback for node 69ce170923eb053795d434f8 INFO: 172.20.0.1:39532 - "POST /node/69ce170923eb053795d434f8 HTTP/1.0" 202 Accepted 2026-04-04 07:13:33,232 - lava_callback - INFO - Starting async job submit for node 69ce170923eb053795d434f8 2026-04-04 07:13:33,233 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:13:33,247 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce170923eb053795d434f8 HTTP/1.1" 200 None 2026-04-04 07:13:33,248 - lava_callback - DEBUG - Processing node 69ce170923eb053795d434f8: baseline-arm-baylibre 2026-04-04 07:13:33,249 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce170923eb053795d434f8 2026-04-04 07:13:33,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:13:33,284 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce170923eb053795d434f8 HTTP/1.1" 200 None 2026-04-04 07:13:33,285 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:13:33,299 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:13:33,300 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:13:33,320 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce170923eb053795d434f8 HTTP/1.1" 200 None 2026-04-04 07:13:33,321 - lava_callback - ERROR - Error processing callback for node 69ce170923eb053795d434f8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42260 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:13:53,562 - lava_callback - INFO - Received callback for node 69ce172d23eb053795d43503 INFO: 172.20.0.1:46170 - "POST /node/69ce172d23eb053795d43503 HTTP/1.0" 202 Accepted 2026-04-04 07:13:53,568 - lava_callback - INFO - Starting async job submit for node 69ce172d23eb053795d43503 2026-04-04 07:13:53,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:13:53,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce172d23eb053795d43503 HTTP/1.1" 200 None 2026-04-04 07:13:53,585 - lava_callback - DEBUG - Processing node 69ce172d23eb053795d43503: baseline-arm-baylibre 2026-04-04 07:13:53,585 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce172d23eb053795d43503 2026-04-04 07:13:53,586 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:13:53,600 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce172d23eb053795d43503 HTTP/1.1" 200 None 2026-04-04 07:13:53,601 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:13:53,615 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:13:53,617 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:13:53,642 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce172d23eb053795d43503 HTTP/1.1" 200 None 2026-04-04 07:13:53,655 - lava_callback - ERROR - Error processing callback for node 69ce172d23eb053795d43503: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46174 - "GET /node/6990ace9c45febfe44521adb%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:46176 - "GET /node/698bcff91e517752bab8ad6c%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:54808 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58844 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58848 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39904 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:14:33,340 - lava_callback - INFO - Received callback for node 69ce175423eb053795d43512 INFO: 172.20.0.1:53220 - "POST /node/69ce175423eb053795d43512 HTTP/1.0" 202 Accepted 2026-04-04 07:14:33,346 - lava_callback - INFO - Starting async job submit for node 69ce175423eb053795d43512 2026-04-04 07:14:33,347 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:14:33,362 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce175423eb053795d43512 HTTP/1.1" 200 None 2026-04-04 07:14:33,363 - lava_callback - DEBUG - Processing node 69ce175423eb053795d43512: baseline-arm-baylibre 2026-04-04 07:14:33,363 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce175423eb053795d43512 2026-04-04 07:14:33,364 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:14:33,378 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce175423eb053795d43512 HTTP/1.1" 200 None 2026-04-04 07:14:33,380 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:14:33,394 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:14:33,395 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:14:33,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce175423eb053795d43512 HTTP/1.1" 200 None 2026-04-04 07:14:33,423 - lava_callback - ERROR - Error processing callback for node 69ce175423eb053795d43512: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53226 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42608 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44856 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47366 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47382 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49170 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51664 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59068 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44088 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56172 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46022 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34152 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34158 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50148 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58608 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58612 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46052 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45620 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54664 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54678 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52236 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46278 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45258 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:20:13,899 - lava_callback - INFO - Received callback for node 69ce18a423eb053795d43549 INFO: 172.20.0.1:45274 - "POST /node/69ce18a423eb053795d43549 HTTP/1.0" 202 Accepted 2026-04-04 07:20:13,908 - lava_callback - INFO - Starting async job submit for node 69ce18a423eb053795d43549 2026-04-04 07:20:13,909 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:20:13,924 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce18a423eb053795d43549 HTTP/1.1" 200 None 2026-04-04 07:20:13,925 - lava_callback - DEBUG - Processing node 69ce18a423eb053795d43549: baseline-arm-baylibre 2026-04-04 07:20:13,925 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce18a423eb053795d43549 2026-04-04 07:20:13,926 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:20:13,950 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce18a423eb053795d43549 HTTP/1.1" 200 None 2026-04-04 07:20:13,953 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:20:13,975 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:20:13,978 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:20:14,011 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce18a423eb053795d43549 HTTP/1.1" 200 None 2026-04-04 07:20:14,012 - lava_callback - ERROR - Error processing callback for node 69ce18a423eb053795d43549: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45278 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45282 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36128 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46024 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:20:53,632 - lava_callback - INFO - Received callback for node 69ce18d123eb053795d43557 INFO: 172.20.0.1:55994 - "POST /node/69ce18d123eb053795d43557 HTTP/1.0" 202 Accepted 2026-04-04 07:20:53,641 - lava_callback - INFO - Starting async job submit for node 69ce18d123eb053795d43557 2026-04-04 07:20:53,642 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:20:53,662 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce18d123eb053795d43557 HTTP/1.1" 200 None 2026-04-04 07:20:53,664 - lava_callback - DEBUG - Processing node 69ce18d123eb053795d43557: baseline-arm64-baylibre 2026-04-04 07:20:53,664 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce18d123eb053795d43557 2026-04-04 07:20:53,665 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:20:53,681 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce18d123eb053795d43557 HTTP/1.1" 200 None 2026-04-04 07:20:53,683 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:20:53,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 07:20:53,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:20:53,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce18d123eb053795d43557 HTTP/1.1" 200 None 2026-04-04 07:20:53,735 - lava_callback - ERROR - Error processing callback for node 69ce18d123eb053795d43557: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60846 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55874 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55890 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:55898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41012 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46826 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60938 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46688 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:22:13,776 - lava_callback - INFO - Received callback for node 69ce191f23eb053795d43561 INFO: 172.20.0.1:46702 - "POST /node/69ce191f23eb053795d43561 HTTP/1.0" 202 Accepted 2026-04-04 07:22:13,782 - lava_callback - INFO - Starting async job submit for node 69ce191f23eb053795d43561 2026-04-04 07:22:13,783 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:22:13,797 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce191f23eb053795d43561 HTTP/1.1" 200 None 2026-04-04 07:22:13,798 - lava_callback - DEBUG - Processing node 69ce191f23eb053795d43561: baseline-arm64-baylibre 2026-04-04 07:22:13,798 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce191f23eb053795d43561 2026-04-04 07:22:13,799 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:22:13,815 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce191f23eb053795d43561 HTTP/1.1" 200 None 2026-04-04 07:22:13,817 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:22:13,833 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 07:22:13,835 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:22:13,856 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce191f23eb053795d43561 HTTP/1.1" 200 None 2026-04-04 07:22:13,860 - lava_callback - ERROR - Error processing callback for node 69ce191f23eb053795d43561: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:46712 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51240 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50640 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48392 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45180 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45182 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45194 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43276 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:23:33,867 - lava_callback - INFO - Received callback for node 69ce196c23eb053795d4356a INFO: 172.20.0.1:39894 - "POST /node/69ce196c23eb053795d4356a HTTP/1.0" 202 Accepted 2026-04-04 07:23:33,872 - lava_callback - INFO - Starting async job submit for node 69ce196c23eb053795d4356a 2026-04-04 07:23:33,873 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:23:33,887 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce196c23eb053795d4356a HTTP/1.1" 200 None 2026-04-04 07:23:33,888 - lava_callback - DEBUG - Processing node 69ce196c23eb053795d4356a: baseline-arm64-baylibre 2026-04-04 07:23:33,889 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce196c23eb053795d4356a 2026-04-04 07:23:33,889 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:23:33,902 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce196c23eb053795d4356a HTTP/1.1" 200 None 2026-04-04 07:23:33,903 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:23:33,916 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 07:23:33,917 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:23:33,938 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce196c23eb053795d4356a HTTP/1.1" 200 None 2026-04-04 07:23:33,946 - lava_callback - ERROR - Error processing callback for node 69ce196c23eb053795d4356a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56868 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40584 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44442 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44446 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59118 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59612 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:24:53,909 - lava_callback - INFO - Received callback for node 69ce19b923eb053795d43574 INFO: 172.20.0.1:38638 - "POST /node/69ce19b923eb053795d43574 HTTP/1.0" 202 Accepted 2026-04-04 07:24:53,917 - lava_callback - INFO - Starting async job submit for node 69ce19b923eb053795d43574 2026-04-04 07:24:53,919 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:24:53,939 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce19b923eb053795d43574 HTTP/1.1" 200 None 2026-04-04 07:24:53,939 - lava_callback - DEBUG - Processing node 69ce19b923eb053795d43574: baseline-arm64-baylibre 2026-04-04 07:24:53,940 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce19b923eb053795d43574 2026-04-04 07:24:53,940 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:24:53,954 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce19b923eb053795d43574 HTTP/1.1" 200 None 2026-04-04 07:24:53,956 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:24:53,970 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 07:24:53,972 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:24:53,996 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce19b923eb053795d43574 HTTP/1.1" 200 None 2026-04-04 07:24:53,998 - lava_callback - ERROR - Error processing callback for node 69ce19b923eb053795d43574: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38642 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59546 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59554 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59556 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37388 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36812 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36404 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36220 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:49544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55310 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41796 - "GET /node/698820f0e31b58f834f1ec57%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41798 - "GET /node/694bd88fc19e09af58d497c1%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41814 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53528 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53540 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40184 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:27:34,069 - lava_callback - INFO - Received callback for node 69ce1a5323eb053795d43587 INFO: 172.20.0.1:38626 - "POST /node/69ce1a5323eb053795d43587 HTTP/1.0" 202 Accepted 2026-04-04 07:27:34,074 - lava_callback - INFO - Starting async job submit for node 69ce1a5323eb053795d43587 2026-04-04 07:27:34,076 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:27:34,090 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1a5323eb053795d43587 HTTP/1.1" 200 None 2026-04-04 07:27:34,091 - lava_callback - DEBUG - Processing node 69ce1a5323eb053795d43587: baseline-arm64-baylibre 2026-04-04 07:27:34,092 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1a5323eb053795d43587 2026-04-04 07:27:34,092 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:27:34,106 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1a5323eb053795d43587 HTTP/1.1" 200 None 2026-04-04 07:27:34,108 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:27:34,123 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 07:27:34,125 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:27:34,145 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1a5323eb053795d43587 HTTP/1.1" 200 None 2026-04-04 07:27:34,151 - lava_callback - ERROR - Error processing callback for node 69ce1a5323eb053795d43587: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:38636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36604 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36610 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36616 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33898 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47636 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53764 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:28:51,169 - lava_callback - INFO - Received callback for node 69ce1aa523eb053795d43596 INFO: 172.20.0.1:57894 - "POST /node/69ce1aa523eb053795d43596 HTTP/1.0" 202 Accepted 2026-04-04 07:28:51,175 - lava_callback - INFO - Starting async job submit for node 69ce1aa523eb053795d43596 2026-04-04 07:28:51,176 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:28:51,190 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1aa523eb053795d43596 HTTP/1.1" 200 None 2026-04-04 07:28:51,190 - lava_callback - DEBUG - Processing node 69ce1aa523eb053795d43596: baseline-arm64-clabbe 2026-04-04 07:28:51,191 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1aa523eb053795d43596 2026-04-04 07:28:51,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:28:51,204 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1aa523eb053795d43596 HTTP/1.1" 200 None 2026-04-04 07:28:51,206 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:28:51,218 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 07:28:51,220 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:28:51,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1aa523eb053795d43596 HTTP/1.1" 200 None 2026-04-04 07:28:51,241 - lava_callback - ERROR - Error processing callback for node 69ce1aa523eb053795d43596: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57904 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48190 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48194 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46146 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40806 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47060 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:30:11,135 - lava_callback - INFO - Received callback for node 69ce1af223eb053795d435a1 INFO: 172.20.0.1:53078 - "POST /node/69ce1af223eb053795d435a1 HTTP/1.0" 202 Accepted 2026-04-04 07:30:11,142 - lava_callback - INFO - Starting async job submit for node 69ce1af223eb053795d435a1 2026-04-04 07:30:11,143 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:30:11,166 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1af223eb053795d435a1 HTTP/1.1" 200 None 2026-04-04 07:30:11,167 - lava_callback - DEBUG - Processing node 69ce1af223eb053795d435a1: baseline-arm64-clabbe 2026-04-04 07:30:11,168 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1af223eb053795d435a1 2026-04-04 07:30:11,169 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:30:11,182 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1af223eb053795d435a1 HTTP/1.1" 200 None 2026-04-04 07:30:11,183 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:30:11,197 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 07:30:11,199 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:30:11,223 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1af223eb053795d435a1 HTTP/1.1" 200 None 2026-04-04 07:30:11,231 - lava_callback - ERROR - Error processing callback for node 69ce1af223eb053795d435a1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:53082 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53084 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53094 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37456 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36010 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52412 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38520 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:38530 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58576 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:31:31,246 - lava_callback - INFO - Received callback for node 69ce1b4023eb053795d435af INFO: 172.20.0.1:56086 - "POST /node/69ce1b4023eb053795d435af HTTP/1.0" 202 Accepted 2026-04-04 07:31:31,253 - lava_callback - INFO - Starting async job submit for node 69ce1b4023eb053795d435af 2026-04-04 07:31:31,253 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:31:31,270 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1b4023eb053795d435af HTTP/1.1" 200 None 2026-04-04 07:31:31,271 - lava_callback - DEBUG - Processing node 69ce1b4023eb053795d435af: baseline-arm64-clabbe 2026-04-04 07:31:31,272 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1b4023eb053795d435af 2026-04-04 07:31:31,272 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:31:31,286 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1b4023eb053795d435af HTTP/1.1" 200 None 2026-04-04 07:31:31,288 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:31:31,312 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 07:31:31,314 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:31:31,335 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1b4023eb053795d435af HTTP/1.1" 200 None 2026-04-04 07:31:31,340 - lava_callback - ERROR - Error processing callback for node 69ce1b4023eb053795d435af: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38942 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56494 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46122 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60080 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:32:31,298 - lava_callback - INFO - Received callback for node 69ce1b8c23eb053795d435bd INFO: 172.20.0.1:35640 - "POST /node/69ce1b8c23eb053795d435bd HTTP/1.0" 202 Accepted 2026-04-04 07:32:31,306 - lava_callback - INFO - Starting async job submit for node 69ce1b8c23eb053795d435bd 2026-04-04 07:32:31,307 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:32:31,321 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1b8c23eb053795d435bd HTTP/1.1" 200 None 2026-04-04 07:32:31,322 - lava_callback - DEBUG - Processing node 69ce1b8c23eb053795d435bd: baseline-arm64-clabbe 2026-04-04 07:32:31,323 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1b8c23eb053795d435bd 2026-04-04 07:32:31,323 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:32:31,340 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1b8c23eb053795d435bd HTTP/1.1" 200 None 2026-04-04 07:32:31,341 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:32:31,360 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 07:32:31,362 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:32:31,395 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1b8c23eb053795d435bd HTTP/1.1" 200 None 2026-04-04 07:32:31,396 - lava_callback - ERROR - Error processing callback for node 69ce1b8c23eb053795d435bd: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44130 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51414 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53456 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53462 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52092 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39800 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59216 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:33:51,439 - lava_callback - INFO - Received callback for node 69ce1bd723eb053795d435c9 INFO: 172.20.0.1:54166 - "POST /node/69ce1bd723eb053795d435c9 HTTP/1.0" 202 Accepted 2026-04-04 07:33:51,444 - lava_callback - INFO - Starting async job submit for node 69ce1bd723eb053795d435c9 2026-04-04 07:33:51,445 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:33:51,460 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1bd723eb053795d435c9 HTTP/1.1" 200 None 2026-04-04 07:33:51,461 - lava_callback - DEBUG - Processing node 69ce1bd723eb053795d435c9: baseline-arm64-clabbe 2026-04-04 07:33:51,461 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1bd723eb053795d435c9 2026-04-04 07:33:51,462 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:33:51,474 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1bd723eb053795d435c9 HTTP/1.1" 200 None 2026-04-04 07:33:51,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:33:51,489 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 07:33:51,491 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:33:51,525 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1bd723eb053795d435c9 HTTP/1.1" 200 None 2026-04-04 07:33:51,531 - lava_callback - ERROR - Error processing callback for node 69ce1bd723eb053795d435c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54178 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35450 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59390 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59406 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35700 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46656 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53766 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52006 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:35:11,363 - lava_callback - INFO - Received callback for node 69ce1c2623eb053795d435da INFO: 172.20.0.1:42354 - "POST /node/69ce1c2623eb053795d435da HTTP/1.0" 202 Accepted 2026-04-04 07:35:11,369 - lava_callback - INFO - Starting async job submit for node 69ce1c2623eb053795d435da 2026-04-04 07:35:11,370 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:35:11,384 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1c2623eb053795d435da HTTP/1.1" 200 None 2026-04-04 07:35:11,385 - lava_callback - DEBUG - Processing node 69ce1c2623eb053795d435da: baseline-arm64-clabbe 2026-04-04 07:35:11,385 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1c2623eb053795d435da 2026-04-04 07:35:11,386 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:35:11,399 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1c2623eb053795d435da HTTP/1.1" 200 None 2026-04-04 07:35:11,400 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:35:11,419 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cdb5cc23eb053795d41ecc HTTP/1.1" 200 None 2026-04-04 07:35:11,420 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:35:11,449 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1c2623eb053795d435da HTTP/1.1" 200 None 2026-04-04 07:35:11,451 - lava_callback - ERROR - Error processing callback for node 69ce1c2623eb053795d435da: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42356 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42364 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42374 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53622 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34910 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37692 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:36:11,447 - lava_callback - INFO - Received callback for node 69ce1c6323eb053795d435e3 INFO: 172.20.0.1:58990 - "POST /node/69ce1c6323eb053795d435e3 HTTP/1.0" 202 Accepted 2026-04-04 07:36:11,453 - lava_callback - INFO - Starting async job submit for node 69ce1c6323eb053795d435e3 2026-04-04 07:36:11,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:36:11,469 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1c6323eb053795d435e3 HTTP/1.1" 200 None 2026-04-04 07:36:11,470 - lava_callback - DEBUG - Processing node 69ce1c6323eb053795d435e3: baseline-arm-clabbe 2026-04-04 07:36:11,470 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1c6323eb053795d435e3 2026-04-04 07:36:11,471 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:36:11,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1c6323eb053795d435e3 HTTP/1.1" 200 None 2026-04-04 07:36:11,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:36:11,513 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:36:11,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:36:11,553 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1c6323eb053795d435e3 HTTP/1.1" 200 None 2026-04-04 07:36:11,558 - lava_callback - ERROR - Error processing callback for node 69ce1c6323eb053795d435e3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58992 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59000 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56496 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42342 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48486 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42106 - "GET /node/6949d4a9d45a0a86fbe2842d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42110 - "GET /node/69410099460bc8e265116900%5Cn HTTP/1.0" 405 Method Not Allowed 2026-04-04 07:36:54,665 - lava_callback - INFO - Received callback for node 69ce1c8723eb053795d435f0 INFO: 172.20.0.1:42116 - "POST /node/69ce1c8723eb053795d435f0 HTTP/1.0" 202 Accepted 2026-04-04 07:36:54,671 - lava_callback - INFO - Starting async job submit for node 69ce1c8723eb053795d435f0 2026-04-04 07:36:54,672 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:36:54,687 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1c8723eb053795d435f0 HTTP/1.1" 200 None 2026-04-04 07:36:54,688 - lava_callback - DEBUG - Processing node 69ce1c8723eb053795d435f0: baseline-arm-baylibre 2026-04-04 07:36:54,688 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1c8723eb053795d435f0 2026-04-04 07:36:54,689 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:36:54,707 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1c8723eb053795d435f0 HTTP/1.1" 200 None 2026-04-04 07:36:54,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:36:54,722 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:36:54,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:36:54,747 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1c8723eb053795d435f0 HTTP/1.1" 200 None 2026-04-04 07:36:54,755 - lava_callback - ERROR - Error processing callback for node 69ce1c8723eb053795d435f0: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42126 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60824 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:37:31,477 - lava_callback - INFO - Received callback for node 69ce1cac23eb053795d43601 INFO: 172.20.0.1:57182 - "POST /node/69ce1cac23eb053795d43601 HTTP/1.0" 202 Accepted 2026-04-04 07:37:31,483 - lava_callback - INFO - Starting async job submit for node 69ce1cac23eb053795d43601 2026-04-04 07:37:31,483 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:37:31,503 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1cac23eb053795d43601 HTTP/1.1" 200 None 2026-04-04 07:37:31,504 - lava_callback - DEBUG - Processing node 69ce1cac23eb053795d43601: baseline-arm-clabbe 2026-04-04 07:37:31,504 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1cac23eb053795d43601 2026-04-04 07:37:31,505 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:37:31,523 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1cac23eb053795d43601 HTTP/1.1" 200 None 2026-04-04 07:37:31,524 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:37:31,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:37:31,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:37:31,576 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1cac23eb053795d43601 HTTP/1.1" 200 None 2026-04-04 07:37:31,577 - lava_callback - ERROR - Error processing callback for node 69ce1cac23eb053795d43601: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:57186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39952 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34014 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:38:11,445 - lava_callback - INFO - Received callback for node 69ce1cd023eb053795d4360f INFO: 172.20.0.1:56822 - "POST /node/69ce1cd023eb053795d4360f HTTP/1.0" 202 Accepted 2026-04-04 07:38:11,451 - lava_callback - INFO - Starting async job submit for node 69ce1cd023eb053795d4360f 2026-04-04 07:38:11,452 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:38:11,467 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1cd023eb053795d4360f HTTP/1.1" 200 None 2026-04-04 07:38:11,468 - lava_callback - DEBUG - Processing node 69ce1cd023eb053795d4360f: baseline-arm-clabbe 2026-04-04 07:38:11,468 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1cd023eb053795d4360f 2026-04-04 07:38:11,469 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:38:11,484 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1cd023eb053795d4360f HTTP/1.1" 200 None 2026-04-04 07:38:11,486 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:38:11,499 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:38:11,500 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:38:11,520 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1cd023eb053795d4360f HTTP/1.1" 200 None 2026-04-04 07:38:11,521 - lava_callback - ERROR - Error processing callback for node 69ce1cd023eb053795d4360f: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56828 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56832 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:56838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51480 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:38:31,524 - lava_callback - INFO - Received callback for node 69ce1cf423eb053795d43627 INFO: 172.20.0.1:50772 - "POST /node/69ce1cf423eb053795d43627 HTTP/1.0" 202 Accepted 2026-04-04 07:38:31,529 - lava_callback - INFO - Starting async job submit for node 69ce1cf423eb053795d43627 2026-04-04 07:38:31,530 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:38:31,545 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1cf423eb053795d43627 HTTP/1.1" 200 None 2026-04-04 07:38:31,546 - lava_callback - DEBUG - Processing node 69ce1cf423eb053795d43627: baseline-arm-clabbe 2026-04-04 07:38:31,546 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1cf423eb053795d43627 2026-04-04 07:38:31,547 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:38:31,570 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1cf423eb053795d43627 HTTP/1.1" 200 None 2026-04-04 07:38:31,572 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:38:31,587 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:38:31,589 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:38:31,614 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1cf423eb053795d43627 HTTP/1.1" 200 None 2026-04-04 07:38:31,616 - lava_callback - ERROR - Error processing callback for node 69ce1cf423eb053795d43627: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59914 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52008 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:39:11,508 - lava_callback - INFO - Received callback for node 69ce1d1923eb053795d4362e INFO: 172.20.0.1:58202 - "POST /node/69ce1d1923eb053795d4362e HTTP/1.0" 202 Accepted 2026-04-04 07:39:11,514 - lava_callback - INFO - Starting async job submit for node 69ce1d1923eb053795d4362e 2026-04-04 07:39:11,515 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:39:11,529 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1d1923eb053795d4362e HTTP/1.1" 200 None 2026-04-04 07:39:11,530 - lava_callback - DEBUG - Processing node 69ce1d1923eb053795d4362e: baseline-arm-clabbe 2026-04-04 07:39:11,530 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1d1923eb053795d4362e 2026-04-04 07:39:11,531 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:39:11,546 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1d1923eb053795d4362e HTTP/1.1" 200 None 2026-04-04 07:39:11,548 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:39:11,568 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:39:11,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:39:11,598 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1d1923eb053795d4362e HTTP/1.1" 200 None 2026-04-04 07:39:11,604 - lava_callback - ERROR - Error processing callback for node 69ce1d1923eb053795d4362e: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58216 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58230 - "GET /node/698bd9671e517752bab8b666%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58246 - "GET /node/694bd3dbc19e09af58d48841%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:58258 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:58264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48730 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41266 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51378 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:39:51,585 - lava_callback - INFO - Received callback for node 69ce1d3d23eb053795d4363b INFO: 172.20.0.1:36478 - "POST /node/69ce1d3d23eb053795d4363b HTTP/1.0" 202 Accepted 2026-04-04 07:39:51,590 - lava_callback - INFO - Starting async job submit for node 69ce1d3d23eb053795d4363b 2026-04-04 07:39:51,591 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:39:51,606 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1d3d23eb053795d4363b HTTP/1.1" 200 None 2026-04-04 07:39:51,606 - lava_callback - DEBUG - Processing node 69ce1d3d23eb053795d4363b: baseline-arm-clabbe 2026-04-04 07:39:51,607 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1d3d23eb053795d4363b 2026-04-04 07:39:51,607 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:39:51,621 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1d3d23eb053795d4363b HTTP/1.1" 200 None 2026-04-04 07:39:51,623 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:39:51,636 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:39:51,638 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:39:51,658 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1d3d23eb053795d4363b HTTP/1.1" 200 None 2026-04-04 07:39:51,670 - lava_callback - ERROR - Error processing callback for node 69ce1d3d23eb053795d4363b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:36488 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36500 - "GET /node/69870f9c6e077e4126a7edc3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41572 - "GET /node/69885206e408e9ea60fadee3%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:41574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48884 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48896 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:48900 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60740 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:40:31,547 - lava_callback - INFO - Received callback for node 69ce1d6323eb053795d4364a INFO: 172.20.0.1:35420 - "POST /node/69ce1d6323eb053795d4364a HTTP/1.0" 202 Accepted 2026-04-04 07:40:31,553 - lava_callback - INFO - Starting async job submit for node 69ce1d6323eb053795d4364a 2026-04-04 07:40:31,554 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:40:31,569 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1d6323eb053795d4364a HTTP/1.1" 200 None 2026-04-04 07:40:31,570 - lava_callback - DEBUG - Processing node 69ce1d6323eb053795d4364a: baseline-arm-clabbe 2026-04-04 07:40:31,570 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1d6323eb053795d4364a 2026-04-04 07:40:31,570 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:40:31,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1d6323eb053795d4364a HTTP/1.1" 200 None 2026-04-04 07:40:31,585 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:40:31,603 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:40:31,604 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:40:31,626 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1d6323eb053795d4364a HTTP/1.1" 200 None 2026-04-04 07:40:31,628 - lava_callback - ERROR - Error processing callback for node 69ce1d6323eb053795d4364a: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:35422 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39498 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34264 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39514 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:41:11,637 - lava_callback - INFO - Received callback for node 69ce1d8723eb053795d43657 INFO: 172.20.0.1:44824 - "POST /node/69ce1d8723eb053795d43657 HTTP/1.0" 202 Accepted 2026-04-04 07:41:11,645 - lava_callback - INFO - Starting async job submit for node 69ce1d8723eb053795d43657 2026-04-04 07:41:11,647 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:41:11,682 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1d8723eb053795d43657 HTTP/1.1" 200 None 2026-04-04 07:41:11,683 - lava_callback - DEBUG - Processing node 69ce1d8723eb053795d43657: baseline-arm-clabbe 2026-04-04 07:41:11,683 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1d8723eb053795d43657 2026-04-04 07:41:11,684 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:41:11,700 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1d8723eb053795d43657 HTTP/1.1" 200 None 2026-04-04 07:41:11,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:41:11,715 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:41:11,716 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:41:11,737 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1d8723eb053795d43657 HTTP/1.1" 200 None 2026-04-04 07:41:11,745 - lava_callback - ERROR - Error processing callback for node 69ce1d8723eb053795d43657: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44838 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44842 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58996 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55320 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:41:51,656 - lava_callback - INFO - Received callback for node 69ce1dab23eb053795d43669 INFO: 172.20.0.1:58180 - "POST /node/69ce1dab23eb053795d43669 HTTP/1.0" 202 Accepted 2026-04-04 07:41:51,662 - lava_callback - INFO - Starting async job submit for node 69ce1dab23eb053795d43669 2026-04-04 07:41:51,663 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:41:51,677 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1dab23eb053795d43669 HTTP/1.1" 200 None 2026-04-04 07:41:51,678 - lava_callback - DEBUG - Processing node 69ce1dab23eb053795d43669: baseline-arm-clabbe 2026-04-04 07:41:51,678 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1dab23eb053795d43669 2026-04-04 07:41:51,679 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:41:51,699 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1dab23eb053795d43669 HTTP/1.1" 200 None 2026-04-04 07:41:51,701 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:41:51,727 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12cb8c06c2799bf55018 HTTP/1.1" 200 None 2026-04-04 07:41:51,729 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:41:51,765 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1dab23eb053795d43669 HTTP/1.1" 200 None 2026-04-04 07:41:51,767 - lava_callback - ERROR - Error processing callback for node 69ce1dab23eb053795d43669: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:58196 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56156 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37896 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:42:14,827 - lava_callback - INFO - Received callback for node 69ce1dd423eb053795d43674 INFO: 172.20.0.1:37910 - "POST /node/69ce1dd423eb053795d43674 HTTP/1.0" 202 Accepted 2026-04-04 07:42:14,833 - lava_callback - INFO - Starting async job submit for node 69ce1dd423eb053795d43674 2026-04-04 07:42:14,833 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:42:14,848 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1dd423eb053795d43674 HTTP/1.1" 200 None 2026-04-04 07:42:14,848 - lava_callback - DEBUG - Processing node 69ce1dd423eb053795d43674: baseline-arm64-baylibre 2026-04-04 07:42:14,849 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1dd423eb053795d43674 2026-04-04 07:42:14,849 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:42:14,871 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1dd423eb053795d43674 HTTP/1.1" 200 None 2026-04-04 07:42:14,872 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:42:14,885 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:42:14,887 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:42:14,908 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1dd423eb053795d43674 HTTP/1.1" 200 None 2026-04-04 07:42:14,910 - lava_callback - ERROR - Error processing callback for node 69ce1dd423eb053795d43674: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:37918 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:37934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57772 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44328 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51376 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51386 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51398 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50544 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:43:34,922 - lava_callback - INFO - Received callback for node 69ce1e2123eb053795d43683 INFO: 172.20.0.1:45638 - "POST /node/69ce1e2123eb053795d43683 HTTP/1.0" 202 Accepted 2026-04-04 07:43:34,928 - lava_callback - INFO - Starting async job submit for node 69ce1e2123eb053795d43683 2026-04-04 07:43:34,929 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:43:34,946 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1e2123eb053795d43683 HTTP/1.1" 200 None 2026-04-04 07:43:34,947 - lava_callback - DEBUG - Processing node 69ce1e2123eb053795d43683: baseline-arm64-baylibre 2026-04-04 07:43:34,947 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1e2123eb053795d43683 2026-04-04 07:43:34,948 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:43:34,964 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1e2123eb053795d43683 HTTP/1.1" 200 None 2026-04-04 07:43:34,965 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:43:34,979 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:43:34,980 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:43:35,007 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1e2123eb053795d43683 HTTP/1.1" 200 None 2026-04-04 07:43:35,008 - lava_callback - ERROR - Error processing callback for node 69ce1e2123eb053795d43683: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:45650 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45666 - "GET /node/6988c4119fc35f969e1ee492%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45668 - "GET /node/6992ea8331166481c8dd8d95%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53434 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51134 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57174 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39562 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:39574 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53890 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51112 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47208 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:47224 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39384 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51292 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39982 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:46:15,022 - lava_callback - INFO - Received callback for node 69ce1eb523eb053795d43695 INFO: 172.20.0.1:39988 - "POST /node/69ce1eb523eb053795d43695 HTTP/1.0" 202 Accepted 2026-04-04 07:46:15,028 - lava_callback - INFO - Starting async job submit for node 69ce1eb523eb053795d43695 2026-04-04 07:46:15,029 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:46:15,049 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1eb523eb053795d43695 HTTP/1.1" 200 None 2026-04-04 07:46:15,051 - lava_callback - DEBUG - Processing node 69ce1eb523eb053795d43695: baseline-arm64-baylibre 2026-04-04 07:46:15,051 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1eb523eb053795d43695 2026-04-04 07:46:15,052 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:46:15,075 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1eb523eb053795d43695 HTTP/1.1" 200 None 2026-04-04 07:46:15,077 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:46:15,092 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:46:15,093 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:46:15,118 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1eb523eb053795d43695 HTTP/1.1" 200 None 2026-04-04 07:46:15,120 - lava_callback - ERROR - Error processing callback for node 69ce1eb523eb053795d43695: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39992 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56204 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57970 - "GET /node/6989379485773790cd34cd48%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57980 - "GET /node/6928b9feacdde99814a24aeb%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57988 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51066 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40298 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35980 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42504 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42508 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43900 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:47:35,200 - lava_callback - INFO - Received callback for node 69ce1f0223eb053795d4369c INFO: 172.20.0.1:56110 - "POST /node/69ce1f0223eb053795d4369c HTTP/1.0" 202 Accepted 2026-04-04 07:47:35,206 - lava_callback - INFO - Starting async job submit for node 69ce1f0223eb053795d4369c 2026-04-04 07:47:35,207 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:47:35,232 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1f0223eb053795d4369c HTTP/1.1" 200 None 2026-04-04 07:47:35,233 - lava_callback - DEBUG - Processing node 69ce1f0223eb053795d4369c: baseline-arm64-baylibre 2026-04-04 07:47:35,233 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1f0223eb053795d4369c 2026-04-04 07:47:35,234 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:47:35,248 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1f0223eb053795d4369c HTTP/1.1" 200 None 2026-04-04 07:47:35,249 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:47:35,263 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:47:35,265 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:47:35,289 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1f0223eb053795d4369c HTTP/1.1" 200 None 2026-04-04 07:47:35,291 - lava_callback - ERROR - Error processing callback for node 69ce1f0223eb053795d4369c: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:56114 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56126 - "GET /node/693b0947fee8383e92b71600%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:56140 - "GET /node/693b1bb115f407b7629e02a7%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:44820 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48248 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43872 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35962 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35976 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:35984 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40998 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34692 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:48:51,987 - lava_callback - INFO - Received callback for node 69ce1f5523eb053795d436ac INFO: 172.20.0.1:34034 - "POST /node/69ce1f5523eb053795d436ac HTTP/1.0" 202 Accepted 2026-04-04 07:48:51,993 - lava_callback - INFO - Starting async job submit for node 69ce1f5523eb053795d436ac 2026-04-04 07:48:51,994 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:48:52,008 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1f5523eb053795d436ac HTTP/1.1" 200 None 2026-04-04 07:48:52,009 - lava_callback - DEBUG - Processing node 69ce1f5523eb053795d436ac: baseline-arm64-clabbe 2026-04-04 07:48:52,009 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1f5523eb053795d436ac 2026-04-04 07:48:52,010 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:48:52,023 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1f5523eb053795d436ac HTTP/1.1" 200 None 2026-04-04 07:48:52,025 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:48:52,038 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:48:52,040 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:48:52,072 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1f5523eb053795d436ac HTTP/1.1" 200 None 2026-04-04 07:48:52,073 - lava_callback - ERROR - Error processing callback for node 69ce1f5523eb053795d436ac: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45690 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45694 - "GET /node/69935d86523507f3806aa83b%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45704 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45708 - "GET /node/6940e1f5460bc8e26511382a%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:45724 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35734 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55344 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45048 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36058 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51380 - "GET /favicon.ico HTTP/1.0" 404 Not Found INFO: 172.20.0.1:51396 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59518 - "GET /login HTTP/1.0" 404 Not Found 2026-04-04 07:50:12,024 - lava_callback - INFO - Received callback for node 69ce1fa223eb053795d436b5 INFO: 172.20.0.1:59526 - "POST /node/69ce1fa223eb053795d436b5 HTTP/1.0" 202 Accepted 2026-04-04 07:50:12,030 - lava_callback - INFO - Starting async job submit for node 69ce1fa223eb053795d436b5 2026-04-04 07:50:12,031 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:50:12,044 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1fa223eb053795d436b5 HTTP/1.1" 200 None 2026-04-04 07:50:12,045 - lava_callback - DEBUG - Processing node 69ce1fa223eb053795d436b5: baseline-arm64-clabbe 2026-04-04 07:50:12,045 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1fa223eb053795d436b5 2026-04-04 07:50:12,046 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:50:12,059 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1fa223eb053795d436b5 HTTP/1.1" 200 None 2026-04-04 07:50:12,060 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:50:12,074 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:50:12,075 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:50:12,095 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1fa223eb053795d436b5 HTTP/1.1" 200 None 2026-04-04 07:50:12,096 - lava_callback - ERROR - Error processing callback for node 69ce1fa223eb053795d436b5: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59532 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59546 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:59552 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53094 - "GET /node/69869d192de8e2dea3666e22%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:53100 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53108 - "GET /node/6985d08671989fb141499b8d%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:42140 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46138 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45948 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43168 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59420 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59424 - "GET /api/metrics HTTP/1.0" 200 OK 2026-04-04 07:51:15,403 - lava_callback - INFO - Received callback for node 69ce1fe823eb053795d436c1 INFO: 172.20.0.1:59426 - "POST /node/69ce1fe823eb053795d436c1 HTTP/1.0" 202 Accepted 2026-04-04 07:51:15,408 - lava_callback - INFO - Starting async job submit for node 69ce1fe823eb053795d436c1 2026-04-04 07:51:15,409 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:51:15,424 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1fe823eb053795d436c1 HTTP/1.1" 200 None 2026-04-04 07:51:15,424 - lava_callback - DEBUG - Processing node 69ce1fe823eb053795d436c1: baseline-arm64-baylibre 2026-04-04 07:51:15,425 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce1fe823eb053795d436c1 2026-04-04 07:51:15,425 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:51:15,439 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce1fe823eb053795d436c1 HTTP/1.1" 200 None 2026-04-04 07:51:15,440 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:51:15,454 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:51:15,455 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:51:15,475 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce1fe823eb053795d436c1 HTTP/1.1" 200 None 2026-04-04 07:51:15,483 - lava_callback - ERROR - Error processing callback for node 69ce1fe823eb053795d436c1: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:59428 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46028 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54970 - "GET /node/69a1be5769d5b6151c03ae26%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:57430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45634 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40006 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40012 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:40016 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41286 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:52:32,146 - lava_callback - INFO - Received callback for node 69ce203823eb053795d436c9 INFO: 172.20.0.1:39154 - "POST /node/69ce203823eb053795d436c9 HTTP/1.0" 202 Accepted 2026-04-04 07:52:32,152 - lava_callback - INFO - Starting async job submit for node 69ce203823eb053795d436c9 2026-04-04 07:52:32,153 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:52:32,169 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce203823eb053795d436c9 HTTP/1.1" 200 None 2026-04-04 07:52:32,170 - lava_callback - DEBUG - Processing node 69ce203823eb053795d436c9: baseline-arm64-clabbe 2026-04-04 07:52:32,170 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce203823eb053795d436c9 2026-04-04 07:52:32,171 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:52:32,189 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce203823eb053795d436c9 HTTP/1.1" 200 None 2026-04-04 07:52:32,191 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:52:32,211 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:52:32,212 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:52:32,234 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce203823eb053795d436c9 HTTP/1.1" 200 None 2026-04-04 07:52:32,235 - lava_callback - ERROR - Error processing callback for node 69ce203823eb053795d436c9: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:39162 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50746 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37602 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42184 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53758 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53764 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:53778 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43518 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55388 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:53:52,215 - lava_callback - INFO - Received callback for node 69ce208223eb053795d436d8 INFO: 172.20.0.1:50630 - "POST /node/69ce208223eb053795d436d8 HTTP/1.0" 202 Accepted 2026-04-04 07:53:52,224 - lava_callback - INFO - Starting async job submit for node 69ce208223eb053795d436d8 2026-04-04 07:53:52,225 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:53:52,240 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce208223eb053795d436d8 HTTP/1.1" 200 None 2026-04-04 07:53:52,241 - lava_callback - DEBUG - Processing node 69ce208223eb053795d436d8: baseline-arm64-clabbe 2026-04-04 07:53:52,241 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce208223eb053795d436d8 2026-04-04 07:53:52,242 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:53:52,261 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce208223eb053795d436d8 HTTP/1.1" 200 None 2026-04-04 07:53:52,263 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:53:52,281 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:53:52,284 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:53:52,344 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce208223eb053795d436d8 HTTP/1.1" 200 None 2026-04-04 07:53:52,346 - lava_callback - ERROR - Error processing callback for node 69ce208223eb053795d436d8: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:50646 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52446 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52450 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:52466 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55400 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58632 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44864 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55606 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51042 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51044 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51054 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36822 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:55:32,239 - lava_callback - INFO - Received callback for node 69ce20de23eb053795d436e6 INFO: 172.20.0.1:47448 - "POST /node/69ce20de23eb053795d436e6 HTTP/1.0" 202 Accepted 2026-04-04 07:55:32,245 - lava_callback - INFO - Starting async job submit for node 69ce20de23eb053795d436e6 2026-04-04 07:55:32,246 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:55:32,265 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce20de23eb053795d436e6 HTTP/1.1" 200 None 2026-04-04 07:55:32,266 - lava_callback - DEBUG - Processing node 69ce20de23eb053795d436e6: baseline-arm64-clabbe 2026-04-04 07:55:32,266 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce20de23eb053795d436e6 2026-04-04 07:55:32,267 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:55:32,293 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce20de23eb053795d436e6 HTTP/1.1" 200 None 2026-04-04 07:55:32,296 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:55:32,315 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:55:32,317 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:55:32,348 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce20de23eb053795d436e6 HTTP/1.1" 200 None 2026-04-04 07:55:32,349 - lava_callback - ERROR - Error processing callback for node 69ce20de23eb053795d436e6: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:47458 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43788 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50932 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47262 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60090 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60102 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:60106 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45740 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48684 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37386 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34568 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:57:12,331 - lava_callback - INFO - Received callback for node 69ce215123eb053795d436f3 INFO: 172.20.0.1:42318 - "POST /node/69ce215123eb053795d436f3 HTTP/1.0" 202 Accepted 2026-04-04 07:57:12,337 - lava_callback - INFO - Starting async job submit for node 69ce215123eb053795d436f3 2026-04-04 07:57:12,338 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:57:12,355 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce215123eb053795d436f3 HTTP/1.1" 200 None 2026-04-04 07:57:12,356 - lava_callback - DEBUG - Processing node 69ce215123eb053795d436f3: baseline-arm64-clabbe 2026-04-04 07:57:12,356 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce215123eb053795d436f3 2026-04-04 07:57:12,356 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:57:12,370 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce215123eb053795d436f3 HTTP/1.1" 200 None 2026-04-04 07:57:12,371 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:57:12,392 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:57:12,394 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:57:12,415 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce215123eb053795d436f3 HTTP/1.1" 200 None 2026-04-04 07:57:12,416 - lava_callback - ERROR - Error processing callback for node 69ce215123eb053795d436f3: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:42334 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42336 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:42350 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:53600 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39074 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42524 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43880 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:57738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33438 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33442 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33444 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43562 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59534 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47044 - "GET / HTTP/1.0" 200 OK 2026-04-04 07:58:52,448 - lava_callback - INFO - Received callback for node 69ce21ac23eb053795d436fe INFO: 172.20.0.1:44898 - "POST /node/69ce21ac23eb053795d436fe HTTP/1.0" 202 Accepted 2026-04-04 07:58:52,457 - lava_callback - INFO - Starting async job submit for node 69ce21ac23eb053795d436fe 2026-04-04 07:58:52,458 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:58:52,495 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce21ac23eb053795d436fe HTTP/1.1" 200 None 2026-04-04 07:58:52,496 - lava_callback - DEBUG - Processing node 69ce21ac23eb053795d436fe: baseline-arm64-clabbe 2026-04-04 07:58:52,496 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce21ac23eb053795d436fe 2026-04-04 07:58:52,497 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:58:52,510 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce21ac23eb053795d436fe HTTP/1.1" 200 None 2026-04-04 07:58:52,512 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:58:52,526 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 07:58:52,528 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 07:58:52,551 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce21ac23eb053795d436fe HTTP/1.1" 200 None 2026-04-04 07:58:52,552 - lava_callback - ERROR - Error processing callback for node 69ce21ac23eb053795d436fe: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:44906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44558 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:44570 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58918 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46182 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:42886 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:39502 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43204 - "GET / HTTP/1.0" 200 OK 2026-04-04 08:00:12,666 - lava_callback - INFO - Received callback for node 69ce220623eb053795d43709 INFO: 172.20.0.1:43566 - "POST /node/69ce220623eb053795d43709 HTTP/1.0" 202 Accepted 2026-04-04 08:00:12,672 - lava_callback - INFO - Starting async job submit for node 69ce220623eb053795d43709 2026-04-04 08:00:12,673 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:00:12,691 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce220623eb053795d43709 HTTP/1.1" 200 None 2026-04-04 08:00:12,692 - lava_callback - DEBUG - Processing node 69ce220623eb053795d43709: baseline-arm64-clabbe 2026-04-04 08:00:12,692 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce220623eb053795d43709 2026-04-04 08:00:12,693 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:00:12,706 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce220623eb053795d43709 HTTP/1.1" 200 None 2026-04-04 08:00:12,708 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:00:12,721 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 08:00:12,723 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:00:12,745 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce220623eb053795d43709 HTTP/1.1" 200 None 2026-04-04 08:00:12,746 - lava_callback - ERROR - Error processing callback for node 69ce220623eb053795d43709: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43578 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43588 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:43596 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36762 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47314 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34728 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43764 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36568 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:36572 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:36576 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40200 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45186 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52966 - "GET / HTTP/1.0" 200 OK 2026-04-04 08:01:52,562 - lava_callback - INFO - Received callback for node 69ce225d23eb053795d43715 INFO: 172.20.0.1:51272 - "POST /node/69ce225d23eb053795d43715 HTTP/1.0" 202 Accepted 2026-04-04 08:01:52,568 - lava_callback - INFO - Starting async job submit for node 69ce225d23eb053795d43715 2026-04-04 08:01:52,569 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:01:52,582 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce225d23eb053795d43715 HTTP/1.1" 200 None 2026-04-04 08:01:52,583 - lava_callback - DEBUG - Processing node 69ce225d23eb053795d43715: baseline-arm64-clabbe 2026-04-04 08:01:52,584 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce225d23eb053795d43715 2026-04-04 08:01:52,584 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:01:52,597 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce225d23eb053795d43715 HTTP/1.1" 200 None 2026-04-04 08:01:52,600 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:01:52,618 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 08:01:52,621 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:01:52,649 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce225d23eb053795d43715 HTTP/1.1" 200 None 2026-04-04 08:01:52,650 - lava_callback - ERROR - Error processing callback for node 69ce225d23eb053795d43715: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:51286 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34698 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54538 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54540 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52036 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47792 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46332 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49306 - "GET /login HTTP/1.0" 404 Not Found INFO: 172.20.0.1:49318 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60840 - "GET / HTTP/1.0" 200 OK 2026-04-04 08:03:12,624 - lava_callback - INFO - Received callback for node 69ce22b423eb053795d43720 INFO: 172.20.0.1:32890 - "POST /node/69ce22b423eb053795d43720 HTTP/1.0" 202 Accepted 2026-04-04 08:03:12,629 - lava_callback - INFO - Starting async job submit for node 69ce22b423eb053795d43720 2026-04-04 08:03:12,630 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:03:12,644 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce22b423eb053795d43720 HTTP/1.1" 200 None 2026-04-04 08:03:12,645 - lava_callback - DEBUG - Processing node 69ce22b423eb053795d43720: baseline-arm64-clabbe 2026-04-04 08:03:12,646 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce22b423eb053795d43720 2026-04-04 08:03:12,646 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:03:12,659 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce22b423eb053795d43720 HTTP/1.1" 200 None 2026-04-04 08:03:12,660 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:03:12,674 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 08:03:12,675 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:03:12,704 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce22b423eb053795d43720 HTTP/1.1" 200 None 2026-04-04 08:03:12,705 - lava_callback - ERROR - Error processing callback for node 69ce22b423eb053795d43720: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:32906 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:32912 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:32928 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56964 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51614 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41062 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48276 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41460 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:41464 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:41480 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:58380 - "GET / HTTP/1.0" 200 OK 2026-04-04 08:04:32,700 - lava_callback - INFO - Received callback for node 69ce230d23eb053795d4372b INFO: 172.20.0.1:34862 - "POST /node/69ce230d23eb053795d4372b HTTP/1.0" 202 Accepted 2026-04-04 08:04:32,706 - lava_callback - INFO - Starting async job submit for node 69ce230d23eb053795d4372b 2026-04-04 08:04:32,707 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:04:32,720 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce230d23eb053795d4372b HTTP/1.1" 200 None 2026-04-04 08:04:32,721 - lava_callback - DEBUG - Processing node 69ce230d23eb053795d4372b: baseline-arm64-clabbe 2026-04-04 08:04:32,721 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce230d23eb053795d4372b 2026-04-04 08:04:32,722 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:04:32,734 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce230d23eb053795d4372b HTTP/1.1" 200 None 2026-04-04 08:04:32,736 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:04:32,749 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 08:04:32,751 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:04:32,771 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce230d23eb053795d4372b HTTP/1.1" 200 None 2026-04-04 08:04:32,772 - lava_callback - ERROR - Error processing callback for node 69ce230d23eb053795d4372b: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:34876 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:46078 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40108 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43290 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33738 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:33746 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:33760 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37408 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43554 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:40422 - "GET / HTTP/1.0" 200 OK 2026-04-04 08:05:52,762 - lava_callback - INFO - Received callback for node 69ce235d23eb053795d43732 INFO: 172.20.0.1:54634 - "POST /node/69ce235d23eb053795d43732 HTTP/1.0" 202 Accepted 2026-04-04 08:05:52,767 - lava_callback - INFO - Starting async job submit for node 69ce235d23eb053795d43732 2026-04-04 08:05:52,768 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:05:52,786 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce235d23eb053795d43732 HTTP/1.1" 200 None 2026-04-04 08:05:52,788 - lava_callback - DEBUG - Processing node 69ce235d23eb053795d43732: baseline-arm64-clabbe 2026-04-04 08:05:52,788 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce235d23eb053795d43732 2026-04-04 08:05:52,789 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:05:52,805 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce235d23eb053795d43732 HTTP/1.1" 200 None 2026-04-04 08:05:52,808 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:05:52,829 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69cb12e08c06c2799bf5502b HTTP/1.1" 200 None 2026-04-04 08:05:52,831 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:05:52,854 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce235d23eb053795d43732 HTTP/1.1" 200 None 2026-04-04 08:05:52,856 - lava_callback - ERROR - Error processing callback for node 69ce235d23eb053795d43732: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:54648 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:43448 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45410 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45424 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:45440 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:52364 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:37080 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:47780 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56244 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51528 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51530 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51544 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55594 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44096 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38862 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:49744 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51408 - "GET /node/6988c2d09fc35f969e1ee162%5Cn HTTP/1.0" 405 Method Not Allowed INFO: 172.20.0.1:51424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34958 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34972 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:34976 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48370 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44430 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51308 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51916 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51924 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51934 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34268 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:44580 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55850 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:45802 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:60032 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54024 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54030 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54040 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50644 - "GET / HTTP/1.0" 200 OK 2026-04-04 08:10:36,468 - lava_callback - INFO - Received callback for node 69ce246923eb053795d43741 INFO: 172.20.0.1:54084 - "POST /node/69ce246923eb053795d43741 HTTP/1.0" 202 Accepted 2026-04-04 08:10:36,475 - lava_callback - INFO - Starting async job submit for node 69ce246923eb053795d43741 2026-04-04 08:10:36,476 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:10:36,491 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce246923eb053795d43741 HTTP/1.1" 200 None 2026-04-04 08:10:36,492 - lava_callback - DEBUG - Processing node 69ce246923eb053795d43741: baseline-arm64-baylibre 2026-04-04 08:10:36,492 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce246923eb053795d43741 2026-04-04 08:10:36,493 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:10:36,516 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce246923eb053795d43741 HTTP/1.1" 200 None 2026-04-04 08:10:36,518 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 INFO: 172.20.0.1:54090 - "GET / HTTP/1.0" 200 OK 2026-04-04 08:10:36,548 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-04 08:10:36,550 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:10:36,584 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce246923eb053795d43741 HTTP/1.1" 200 None 2026-04-04 08:10:36,589 - lava_callback - ERROR - Error processing callback for node 69ce246923eb053795d43741: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:41424 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:50742 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:34034 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51692 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:51706 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:51714 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:56492 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:55946 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:35962 - "GET / HTTP/1.0" 200 OK 2026-04-04 08:11:53,013 - lava_callback - INFO - Received callback for node 69ce24c123eb053795d43753 INFO: 172.20.0.1:43166 - "POST /node/69ce24c123eb053795d43753 HTTP/1.0" 202 Accepted 2026-04-04 08:11:53,020 - lava_callback - INFO - Starting async job submit for node 69ce24c123eb053795d43753 2026-04-04 08:11:53,021 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:11:53,045 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce24c123eb053795d43753 HTTP/1.1" 200 None 2026-04-04 08:11:53,046 - lava_callback - DEBUG - Processing node 69ce24c123eb053795d43753: baseline-arm64-clabbe 2026-04-04 08:11:53,046 - lava_callback - WARNING - No log data found in callback Job failure metadata not found for node: 69ce24c123eb053795d43753 2026-04-04 08:11:53,047 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:11:53,063 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69ce24c123eb053795d43753 HTTP/1.1" 200 None 2026-04-04 08:11:53,064 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:11:53,082 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "GET /latest/node/69c86f7ef41e4fb6a3782f4d HTTP/1.1" 200 None 2026-04-04 08:11:53,084 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): staging.kernelci.org:9000 2026-04-04 08:11:53,115 - urllib3.connectionpool - DEBUG - https://staging.kernelci.org:9000 "PUT /latest/nodes/69ce24c123eb053795d43753 HTTP/1.1" 200 None 2026-04-04 08:11:53,117 - lava_callback - ERROR - Error processing callback for node 69ce24c123eb053795d43753: 'lava' Traceback (most recent call last): File "/home/kernelci/pipeline/lava_callback.py", line 366, in async_job_submit _emit_callback_telemetry( File "/home/kernelci/pipeline/lava_callback.py", line 387, in _emit_callback_telemetry is_infra = job_callback.is_infra_error() if job_node.get( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/site-packages/kernelci/runtime/lava.py", line 119, in is_infra_error lava_yaml = self._data['results']['lava'] ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^ KeyError: 'lava' INFO: 172.20.0.1:43176 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:38132 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54206 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:54216 - "GET /api/metrics HTTP/1.0" 200 OK INFO: 172.20.0.1:54218 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:48542 - "GET / HTTP/1.0" 200 OK INFO: 172.20.0.1:59110 - "GET / HTTP/1.0" 200 OK INFO: Shutting down INFO: Waiting for application shutdown. INFO: Application shutdown complete. INFO: Finished server process [1]