Package evaluation of RustyObjectStore on Julia 1.13.0-DEV.139 (5a3c7578c0*) started at 2025-03-05T22:46:19.931 ################################################################################ # Set-up # Installing PkgEval dependencies (TestEnv)... Set-up completed after 8.71s ################################################################################ # Installation # Installing RustyObjectStore... Resolving package versions... Installed SimpleBufferStream ─── v1.2.0 Installed BitFlags ───────────── v0.1.9 Installed LoggingExtras ──────── v1.1.0 Installed Parsers ────────────── v2.8.1 Installed JSON3 ──────────────── v1.14.1 Installed minio_jll ──────────── v1.0.0+1 Installed azurite_jll ────────── v3.29.0+0 Installed Preferences ────────── v1.4.3 Installed ConcurrentUtilities ── v2.5.0 Installed JLLWrappers ────────── v1.7.0 Installed OpenSSL ────────────── v1.4.3 Installed PrecompileTools ────── v1.2.1 Installed URIs ───────────────── v1.5.1 Installed MD5 ────────────────── v0.2.3 Installed TranscodingStreams ─── v0.11.3 Installed CloudBase ──────────── v1.4.7 Installed FunctionWrappers ───── v1.1.3 Installed HTTP ───────────────── v1.10.15 Installed MbedTLS_jll ────────── v2.28.6+2 Installed MbedTLS ────────────── v1.1.9 Installed object_store_ffi_jll ─ v0.12.3+0 Installed CodecZlib ──────────── v0.7.8 Installed RustyObjectStore ───── v0.12.3 Installed NodeJS_16_jll ──────── v16.15.0+0 Installed StructTypes ────────── v1.11.0 Installed DocStringExtensions ── v0.9.3 Installed ExceptionUnwrapping ── v0.1.11 Installed Figgy ──────────────── v1.1.1 Installing 5 artifacts Installed artifact MbedTLS 2.1 MiB Installed artifact azurite 15.7 MiB Installed artifact object_store_ffi 23.8 MiB Installed artifact NodeJS_16 31.2 MiB Installed artifact minio 48.7 MiB Updating `~/.julia/environments/v1.13/Project.toml` [1b5eed3d] + RustyObjectStore v0.12.3 Updating `~/.julia/environments/v1.13/Manifest.toml` [d1d4a3ce] + BitFlags v0.1.9 [85eb1798] + CloudBase v1.4.7 [944b1d66] + CodecZlib v0.7.8 [f0e56b4a] + ConcurrentUtilities v2.5.0 [ffbed154] + DocStringExtensions v0.9.3 [460bff9d] + ExceptionUnwrapping v0.1.11 [937e9308] + Figgy v1.1.1 [069b7b12] + FunctionWrappers v1.1.3 [cd3eb016] + HTTP v1.10.15 [692b3bcd] + JLLWrappers v1.7.0 [0f8b85d8] + JSON3 v1.14.1 [e6f89c97] + LoggingExtras v1.1.0 [6ac74813] + MD5 v0.2.3 [739be429] + MbedTLS v1.1.9 [4d8831e6] + OpenSSL v1.4.3 [69de0a69] + Parsers v2.8.1 [aea7be01] + PrecompileTools v1.2.1 [21216c6a] + Preferences v1.4.3 [1b5eed3d] + RustyObjectStore v0.12.3 [777ac1f9] + SimpleBufferStream v1.2.0 [856f2bd8] + StructTypes v1.11.0 [3bb67fe8] + TranscodingStreams v0.11.3 [5c2747f8] + URIs v1.5.1 [c8ffd9c3] + MbedTLS_jll v2.28.6+2 [a4b94fbf] + NodeJS_16_jll v16.15.0+0 [d59cea07] + azurite_jll v3.29.0+0 [ad83bbfc] + minio_jll v1.0.0+1 [0e112785] + object_store_ffi_jll v0.12.3+0 [0dad84c5] + ArgTools v1.1.2 [56f22d72] + Artifacts v1.11.0 [2a0f44e3] + Base64 v1.11.0 [ade2ca70] + Dates v1.11.0 [f43a241f] + Downloads v1.7.0 [7b1f6079] + FileWatching v1.11.0 [b77e0a4c] + InteractiveUtils v1.11.0 [ac6e5ff7] + JuliaSyntaxHighlighting v1.12.0 [b27032c2] + LibCURL v0.6.4 [76f85450] + LibGit2 v1.11.0 [8f399da3] + Libdl v1.11.0 [56ddb016] + Logging v1.11.0 [d6f4376e] + Markdown v1.11.0 [a63ad114] + Mmap v1.11.0 [ca575930] + NetworkOptions v1.3.0 [44cfe95a] + Pkg v1.12.0 [de0858da] + Printf v1.11.0 [9a3f8284] + Random v1.11.0 [ea8e919c] + SHA v0.7.0 [9e88b42a] + Serialization v1.11.0 [6462fe0b] + Sockets v1.11.0 [f489334b] + StyledStrings v1.11.0 [fa267f1f] + TOML v1.0.3 [a4e569a6] + Tar v1.10.0 [8dfed614] + Test v1.11.0 [cf7118a7] + UUIDs v1.11.0 [4ec0a83e] + Unicode v1.11.0 [deac9b47] + LibCURL_jll v8.11.1+1 [e37daf67] + LibGit2_jll v1.9.0+0 [29816b5a] + LibSSH2_jll v1.11.3+1 [14a3606d] + MozillaCACerts_jll v2024.12.31 [458c3c95] + OpenSSL_jll v3.0.16+0 [83775a58] + Zlib_jll v1.3.1+2 [8e850ede] + nghttp2_jll v1.64.0+1 [3f19e933] + p7zip_jll v17.5.0+2 Installation completed after 36.9s ################################################################################ # Precompilation # Precompiling PkgEval dependencies... Precompiling packages... 6556.4 ms ✓ TestEnv 1 dependency successfully precompiled in 9 seconds. 25 already precompiled. Precompiling package dependencies... Precompilation completed after 152.52s ################################################################################ # Testing # Testing RustyObjectStore Status `/tmp/jl_mAIgtM/Project.toml` [85eb1798] CloudBase v1.4.7 [944b1d66] CodecZlib v0.7.8 [ffbed154] DocStringExtensions v0.9.3 [cd3eb016] HTTP v1.10.15 [0f8b85d8] JSON3 v1.14.1 [817f1d60] ReTestItems v1.29.0 [1b5eed3d] RustyObjectStore v0.12.3 [0e112785] object_store_ffi_jll v0.12.3+0 [2a0f44e3] Base64 v1.11.0 [6462fe0b] Sockets v1.11.0 [8dfed614] Test v1.11.0 Status `/tmp/jl_mAIgtM/Manifest.toml` [d1d4a3ce] BitFlags v0.1.9 [85eb1798] CloudBase v1.4.7 [944b1d66] CodecZlib v0.7.8 [f0e56b4a] ConcurrentUtilities v2.5.0 [ffbed154] DocStringExtensions v0.9.3 [460bff9d] ExceptionUnwrapping v0.1.11 [937e9308] Figgy v1.1.1 [069b7b12] FunctionWrappers v1.1.3 [cd3eb016] HTTP v1.10.15 [692b3bcd] JLLWrappers v1.7.0 [0f8b85d8] JSON3 v1.14.1 [e6f89c97] LoggingExtras v1.1.0 [6ac74813] MD5 v0.2.3 [739be429] MbedTLS v1.1.9 [4d8831e6] OpenSSL v1.4.3 [69de0a69] Parsers v2.8.1 [aea7be01] PrecompileTools v1.2.1 [21216c6a] Preferences v1.4.3 [817f1d60] ReTestItems v1.29.0 [1b5eed3d] RustyObjectStore v0.12.3 [777ac1f9] SimpleBufferStream v1.2.0 [856f2bd8] StructTypes v1.11.0 [1e6cf692] TestEnv v1.102.0 [3bb67fe8] TranscodingStreams v0.11.3 [5c2747f8] URIs v1.5.1 [c8ffd9c3] MbedTLS_jll v2.28.6+2 [a4b94fbf] NodeJS_16_jll v16.15.0+0 [d59cea07] azurite_jll v3.29.0+0 [ad83bbfc] minio_jll v1.0.0+1 [0e112785] object_store_ffi_jll v0.12.3+0 [0dad84c5] ArgTools v1.1.2 [56f22d72] Artifacts v1.11.0 [2a0f44e3] Base64 v1.11.0 [ade2ca70] Dates v1.11.0 [f43a241f] Downloads v1.7.0 [7b1f6079] FileWatching v1.11.0 [b77e0a4c] InteractiveUtils v1.11.0 [ac6e5ff7] JuliaSyntaxHighlighting v1.12.0 [b27032c2] LibCURL v0.6.4 [76f85450] LibGit2 v1.11.0 [8f399da3] Libdl v1.11.0 [56ddb016] Logging v1.11.0 [d6f4376e] Markdown v1.11.0 [a63ad114] Mmap v1.11.0 [ca575930] NetworkOptions v1.3.0 [44cfe95a] Pkg v1.12.0 [de0858da] Printf v1.11.0 [9a3f8284] Random v1.11.0 [ea8e919c] SHA v0.7.0 [9e88b42a] Serialization v1.11.0 [6462fe0b] Sockets v1.11.0 [f489334b] StyledStrings v1.11.0 [fa267f1f] TOML v1.0.3 [a4e569a6] Tar v1.10.0 [8dfed614] Test v1.11.0 [cf7118a7] UUIDs v1.11.0 [4ec0a83e] Unicode v1.11.0 [deac9b47] LibCURL_jll v8.11.1+1 [e37daf67] LibGit2_jll v1.9.0+0 [29816b5a] LibSSH2_jll v1.11.3+1 [14a3606d] MozillaCACerts_jll v2024.12.31 [458c3c95] OpenSSL_jll v3.0.16+0 [83775a58] Zlib_jll v1.3.1+2 [8e850ede] nghttp2_jll v1.64.0+1 [3f19e933] p7zip_jll v17.5.0+2 Testing Running tests... ┌ Warning: Verbosity logging macros are deprecated as they are not compatible with juliac-compiled programs │ caller = ip:0x0 └ @ Core :-1 [ Info: Scanning for test items in project `RustyObjectStore` at paths: /home/pkgeval/.julia/packages/RustyObjectStore/JF6aG [ Info: Finished scanning for test items in 21.65 seconds. Scheduling 20 tests on pid 174 with 1 worker processes and 2 threads per worker. [ Info: Starting test workers Worker 179: [ Info: Starting test worker 1 on pid = 179, with 2 threads [ Info: Starting running test items Worker 179: 22:51:30 | maxrss 2.4% | mem 4.8% | START ( 1/20) test item "AWSConfig" at test/aws_api_tests.jl:1 Worker 179: ┌ Warning: Verbosity logging macros are deprecated as they are not compatible with juliac-compiled programs Worker 179: │ caller = ip:0x0 Worker 179: └ @ Core :-1 Worker 179: 22:51:40 | maxrss 2.4% | mem 5.1% | DONE ( 1/20) test item "AWSConfig" 6.9 secs (71.0% compile, 13.9% recompile, 1.3% GC), 2.99 M allocs (175.989 MB) Worker 179: 22:52:11 | maxrss 2.4% | mem 5.2% | START ( 2/20) test item "Basic S3 exceptions" at test/aws_s3_exception_tests.jl:1 Worker 179: 22:53:00 | maxrss 2.5% | mem 5.7% | DONE ( 2/20) test item "Basic S3 exceptions" 47.8 secs (89.3% compile, 0.3% recompile, 1.2% GC), 16.42 M allocs (950.701 MB) ┌ Warning: Test item "Basic S3 exceptions" at test/aws_s3_exception_tests.jl:1 contains test sets without tests: │ "Delete non-existing file" └ @ ReTestItems ~/.julia/packages/ReTestItems/ecp6j/src/log_capture.jl:324 Worker 179: 22:54:23 | maxrss 2.5% | mem 5.6% | START ( 3/20) test item "AWS S3 retries" at test/aws_s3_exception_tests.jl:159 Worker 179: 22:55:05 | maxrss 2.5% | mem 5.7% | DONE ( 3/20) test item "AWS S3 retries" 40.7 secs (26.9% compile, 1.9% recompile, 0.3% GC), 5.81 M allocs (349.319 MB) Captured Logs for test item "AWS S3 retries" at test/aws_s3_exception_tests.jl:159 on worker 179 2025-03-05T22:54:28.450034Z  WARN object_store_ffi: Generic S3 error: Client error with status 400 Bad Request: response body from the dummy server Caused by: 0: Generic S3 error: Client error with status 400 Bad Request: response body from the dummy server 1: Client error with status 400 Bad Request: response body from the dummy server Recent attempts (1 out of 1): reason: Code(400) after 0 retries Total retries: 0 Total Time: 1.826877199s 2025-03-05T22:54:29.081355Z  WARN object_store_ffi: Generic S3 error: Client error with status 400 Bad Request: response body from the dummy server Caused by: 0: Generic S3 error: Client error with status 400 Bad Request: response body from the dummy server 1: Client error with status 400 Bad Request: response body from the dummy server Recent attempts (1 out of 1): reason: Code(400) after 0 retries Total retries: 0 Total Time: 1.059829ms 2025-03-05T22:54:29.661175Z  WARN object_store_ffi: The operation lacked the necessary privileges to complete for path blob: Client error with status 403 Forbidden: response body from the dummy server Caused by: 0: The operation lacked the necessary privileges to complete for path blob: Client error with status 403 Forbidden: response body from the dummy server 1: Client error with status 403 Forbidden: response body from the dummy server Recent attempts (1 out of 1): reason: Code(403) after 0 retries Total retries: 0 Total Time: 926.691µs 2025-03-05T22:54:30.245260Z  WARN object_store_ffi: The operation lacked the necessary privileges to complete for path blob: Client error with status 403 Forbidden: response body from the dummy server Caused by: 0: The operation lacked the necessary privileges to complete for path blob: Client error with status 403 Forbidden: response body from the dummy server 1: Client error with status 403 Forbidden: response body from the dummy server Recent attempts (1 out of 1): reason: Code(403) after 0 retries Total retries: 0 Total Time: 998.501µs 2025-03-05T22:54:30.810244Z  WARN object_store_ffi: Object at location blob not found: Client error with status 404 Not Found: response body from the dummy server Caused by: 0: Object at location blob not found: Client error with status 404 Not Found: response body from the dummy server 1: Client error with status 404 Not Found: response body from the dummy server Recent attempts (1 out of 1): reason: Code(404) after 0 retries Total retries: 0 Total Time: 1.04169ms 2025-03-05T22:54:32.090409Z  WARN object_store_ffi: Generic S3 error: Client error with status 405 Method Not Allowed: response body from the dummy server Caused by: 0: Generic S3 error: Client error with status 405 Method Not Allowed: response body from the dummy server 1: Client error with status 405 Method Not Allowed: response body from the dummy server Recent attempts (1 out of 1): reason: Code(405) after 0 retries Total retries: 0 Total Time: 219.144675ms 2025-03-05T22:54:32.686528Z  WARN object_store_ffi: Generic S3 error: Client error with status 405 Method Not Allowed: response body from the dummy server Caused by: 0: Generic S3 error: Client error with status 405 Method Not Allowed: response body from the dummy server 1: Client error with status 405 Method Not Allowed: response body from the dummy server Recent attempts (1 out of 1): reason: Code(405) after 0 retries Total retries: 0 Total Time: 1.252407ms 2025-03-05T22:54:33.237294Z  WARN object_store_ffi: Object at location blob already exists: Client error with status 409 Conflict: response body from the dummy server Caused by: 0: Object at location blob already exists: Client error with status 409 Conflict: response body from the dummy server 1: Client error with status 409 Conflict: response body from the dummy server Recent attempts (1 out of 1): reason: Code(409) after 0 retries Total retries: 0 Total Time: 1.01842ms 2025-03-05T22:54:33.822213Z  WARN object_store_ffi: Object at location blob already exists: Client error with status 409 Conflict: response body from the dummy server Caused by: 0: Object at location blob already exists: Client error with status 409 Conflict: response body from the dummy server 1: Client error with status 409 Conflict: response body from the dummy server Recent attempts (1 out of 1): reason: Code(409) after 0 retries Total retries: 0 Total Time: 948.91µs 2025-03-05T22:54:34.377327Z  WARN object_store_ffi: Request precondition failure for path blob: Client error with status 412 Precondition Failed: response body from the dummy server Caused by: 0: Request precondition failure for path blob: Client error with status 412 Precondition Failed: response body from the dummy server 1: Client error with status 412 Precondition Failed: response body from the dummy server Recent attempts (1 out of 1): reason: Code(412) after 0 retries Total retries: 0 Total Time: 1.044209ms 2025-03-05T22:54:34.958158Z  WARN object_store_ffi: Request precondition failure for path blob: Client error with status 412 Precondition Failed: response body from the dummy server Caused by: 0: Request precondition failure for path blob: Client error with status 412 Precondition Failed: response body from the dummy server 1: Client error with status 412 Precondition Failed: response body from the dummy server Recent attempts (1 out of 1): reason: Code(412) after 0 retries Total retries: 0 Total Time: 920.921µs 2025-03-05T22:54:35.512111Z  WARN object_store_ffi: Generic S3 error: Client error with status 413 Payload Too Large: response body from the dummy server Caused by: 0: Generic S3 error: Client error with status 413 Payload Too Large: response body from the dummy server 1: Client error with status 413 Payload Too Large: response body from the dummy server Recent attempts (1 out of 1): reason: Code(413) after 0 retries Total retries: 0 Total Time: 882.861µs 2025-03-05T22:54:36.541106Z  WARN object_store_ffi: Generic S3 error: Client error with status 429 Too Many Requests: response body from the dummy server Caused by: 0: Generic S3 error: Client error with status 429 Too Many Requests: response body from the dummy server 1: Client error with status 429 Too Many Requests: response body from the dummy server Recent attempts (1 out of 1): reason: Code(429) after 0 retries Total retries: 0 Total Time: 917.971µs 2025-03-05T22:54:37.122358Z  WARN object_store_ffi: Generic S3 error: Client error with status 429 Too Many Requests: response body from the dummy server Caused by: 0: Generic S3 error: Client error with status 429 Too Many Requests: response body from the dummy server 1: Client error with status 429 Too Many Requests: response body from the dummy server Recent attempts (1 out of 1): reason: Code(429) after 0 retries Total retries: 0 Total Time: 1.05205ms 2025-03-05T22:54:38.389722Z  WARN object_store_ffi: Generic S3 error: Client error with status 429 Too Many Requests: response body from the dummy server Caused by: 0: Generic S3 error: Client error with status 429 Too Many Requests: response body from the dummy server 1: Client error with status 429 Too Many Requests: response body from the dummy server Recent attempts (1 out of 1): reason: Code(429) after 0 retries Total retries: 0 Total Time: 241.468371ms 2025-03-05T22:54:39.112172Z  WARN object_store_ffi: Generic S3 error: Client error with status 429 Too Many Requests: response body from the dummy server Caused by: 0: Generic S3 error: Client error with status 429 Too Many Requests: response body from the dummy server 1: Client error with status 429 Too Many Requests: response body from the dummy server Recent attempts (1 out of 1): reason: Code(429) after 0 retries Total retries: 0 Total Time: 1.085709ms 2025-03-05T22:54:40.037844Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 278.138774ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (502 Bad Gateway) for url (http://127.0.0.1:8081/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 278.138774ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (502 Bad Gateway) for url (http://127.0.0.1:8081/mybucket/blob) 1: Error after 2 retries in 278.138774ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (502 Bad Gateway) for url (http://127.0.0.1:8081/mybucket/blob) 2: HTTP status server error (502 Bad Gateway) for url (http://127.0.0.1:8081/mybucket/blob) Recent attempts (1 out of 1): reason: Code(502) after 2 retries Total retries: 2 Total Time: 278.407272ms 2025-03-05T22:54:40.846300Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 206.873578ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (502 Bad Gateway) for url (http://127.0.0.1:8081/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 206.873578ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (502 Bad Gateway) for url (http://127.0.0.1:8081/mybucket/blob) 1: Error after 2 retries in 206.873578ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (502 Bad Gateway) for url (http://127.0.0.1:8081/mybucket/blob) 2: HTTP status server error (502 Bad Gateway) for url (http://127.0.0.1:8081/mybucket/blob) Recent attempts (1 out of 1): reason: Code(502) after 2 retries Total retries: 2 Total Time: 207.051936ms 2025-03-05T22:54:41.640505Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 235.080515ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (503 Service Unavailable) for url (http://127.0.0.1:8081/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 235.080515ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (503 Service Unavailable) for url (http://127.0.0.1:8081/mybucket/blob) 1: Error after 2 retries in 235.080515ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (503 Service Unavailable) for url (http://127.0.0.1:8081/mybucket/blob) 2: HTTP status server error (503 Service Unavailable) for url (http://127.0.0.1:8081/mybucket/blob) Recent attempts (1 out of 1): reason: Code(503) after 2 retries Total retries: 2 Total Time: 235.252254ms 2025-03-05T22:54:42.487615Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 239.233224ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (503 Service Unavailable) for url (http://127.0.0.1:8081/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 239.233224ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (503 Service Unavailable) for url (http://127.0.0.1:8081/mybucket/blob) 1: Error after 2 retries in 239.233224ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (503 Service Unavailable) for url (http://127.0.0.1:8081/mybucket/blob) 2: HTTP status server error (503 Service Unavailable) for url (http://127.0.0.1:8081/mybucket/blob) Recent attempts (1 out of 1): reason: Code(503) after 2 retries Total retries: 2 Total Time: 239.404952ms 2025-03-05T22:54:43.335499Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 273.128724ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (504 Gateway Timeout) for url (http://127.0.0.1:8081/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 273.128724ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (504 Gateway Timeout) for url (http://127.0.0.1:8081/mybucket/blob) 1: Error after 2 retries in 273.128724ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (504 Gateway Timeout) for url (http://127.0.0.1:8081/mybucket/blob) 2: HTTP status server error (504 Gateway Timeout) for url (http://127.0.0.1:8081/mybucket/blob) Recent attempts (1 out of 1): reason: Code(504) after 2 retries Total retries: 2 Total Time: 273.291592ms 2025-03-05T22:54:44.218513Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 288.059964ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (504 Gateway Timeout) for url (http://127.0.0.1:8081/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 288.059964ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (504 Gateway Timeout) for url (http://127.0.0.1:8081/mybucket/blob) 1: Error after 2 retries in 288.059964ms, max_retries:2, retry_timeout:10s, source:HTTP status server error (504 Gateway Timeout) for url (http://127.0.0.1:8081/mybucket/blob) 2: HTTP status server error (504 Gateway Timeout) for url (http://127.0.0.1:8081/mybucket/blob) Recent attempts (1 out of 1): reason: Code(504) after 2 retries Total retries: 2 Total Time: 288.244932ms 2025-03-05T22:54:48.435113Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 3.215984113s, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8081/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 3.215984113s, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8081/mybucket/blob) 1: Error after 2 retries in 3.215984113s, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8081/mybucket/blob) 2: error sending request for url (http://127.0.0.1:8081/mybucket/blob) 3: operation timed out Recent attempts (1 out of 1): reason: Timeout after 2 retries Total retries: 2 Total Time: 3.216139301s 2025-03-05T22:54:52.401094Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 3.241649886s, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8081/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 3.241649886s, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8081/mybucket/blob) 1: Error after 2 retries in 3.241649886s, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8081/mybucket/blob) 2: error sending request for url (http://127.0.0.1:8081/mybucket/blob) 3: operation timed out Recent attempts (1 out of 1): reason: Timeout after 2 retries Total retries: 2 Total Time: 3.241825944s 2025-03-05T22:54:54.215383Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 426.864073ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 426.864073ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) 1: Error after 2 retries in 426.864073ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) 2: error sending request for url (http://127.0.0.1:8082/mybucket/blob) 3: client error (SendRequest) 4: connection closed before message completed Recent attempts (1 out of 1): reason: Io after 2 retries Total retries: 2 Total Time: 427.033182ms 2025-03-05T22:54:54.550006Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 307.195632ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 307.195632ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) 1: Error after 2 retries in 307.195632ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) 2: error sending request for url (http://127.0.0.1:8082/mybucket/blob) 3: client error (SendRequest) 4: connection closed before message completed Recent attempts (1 out of 1): reason: Io after 2 retries Total retries: 2 Total Time: 307.383651ms 2025-03-05T22:54:55.105212Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 303.084874ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 303.084874ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) 1: Error after 2 retries in 303.084874ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) 2: error sending request for url (http://127.0.0.1:8082/mybucket/blob) 3: client error (SendRequest) 4: connection error 5: Connection reset by peer (os error 104) Recent attempts (1 out of 1): reason: Io after 2 retries Total retries: 2 Total Time: 303.267252ms 2025-03-05T22:54:55.441356Z  WARN object_store_ffi: Generic S3 error: Error after 2 retries in 304.751017ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) Caused by: 0: Generic S3 error: Error after 2 retries in 304.751017ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) 1: Error after 2 retries in 304.751017ms, max_retries:2, retry_timeout:10s, source:error sending request for url (http://127.0.0.1:8082/mybucket/blob) 2: error sending request for url (http://127.0.0.1:8082/mybucket/blob) 3: client error (SendRequest) 4: connection error 5: Connection reset by peer (os error 104) Recent attempts (1 out of 1): reason: Io after 2 retries Total retries: 2 Total Time: 304.962265ms 2025-03-05T22:54:55.881371Z  WARN object_store_ffi::crud_ops: Error while reading body: io error while streaming body: Generic S3 error: error decoding response body 2025-03-05T22:54:56.459369Z  WARN object_store_ffi::crud_ops: Error while reading body: io error while streaming body: Generic S3 error: error decoding response body 2025-03-05T22:54:56.958194Z  WARN object_store_ffi::crud_ops: Error while reading body: io error while streaming body: Generic S3 error: error decoding response body 2025-03-05T22:54:56.958289Z  WARN object_store_ffi: io error while streaming body: Generic S3 error: error decoding response body Caused by: 0: Generic S3 error: error decoding response body 1: error decoding response body 2: request or response body error 3: error reading a body from connection 4: end of file before message length reached Recent attempts (3 out of 3): reason: Io after 0 retries reason: Io after 0 retries reason: Io after 0 retries Total retries: 2 Total Time: 1.18382991s 2025-03-05T22:54:57.414377Z  WARN object_store_ffi::crud_ops: Error while reading body: io error while streaming body: Generic S3 error: error decoding response body 2025-03-05T22:54:57.781031Z  WARN object_store_ffi::crud_ops: Error while reading body: io error while streaming body: Generic S3 error: error decoding response body 2025-03-05T22:54:58.832239Z  WARN object_store_ffi::crud_ops: Error while reading body: io error while streaming body: Generic S3 error: error decoding response body 2025-03-05T22:54:58.832400Z  WARN object_store_ffi: io error while streaming body: Generic S3 error: error decoding response body Caused by: 0: Generic S3 error: error decoding response body 1: error decoding response body 2: request or response body error 3: error reading a body from connection 4: Connection reset by peer (os error 104) Recent attempts (3 out of 3): reason: Io after 0 retries reason: Io after 0 retries reason: Io after 0 retries Total retries: 2 Total Time: 1.52725405s 2025-03-05T22:55:00.310684Z  WARN object_store_ffi::crud_ops: Error while reading body: io error while streaming body: Generic S3 error: error decoding response body 2025-03-05T22:55:01.794014Z  WARN object_store_ffi::crud_ops: Error while reading body: io error while streaming body: Generic S3 error: error decoding response body 2025-03-05T22:55:03.287350Z  WARN object_store_ffi::crud_ops: Error while reading body: io error while streaming body: Generic S3 error: error decoding response body 2025-03-05T22:55:03.287435Z  WARN object_store_ffi: io error while streaming body: Generic S3 error: error decoding response body Caused by: 0: Generic S3 error: error decoding response body 1: error decoding response body 2: request or response body error 3: operation timed out Recent attempts (3 out of 3): reason: Timeout after 0 retries reason: Timeout after 0 retries reason: Timeout after 0 retries Total retries: 2 Total Time: 3.97783245s 2025-03-05T22:55:04.660537Z  WARN object_store_ffi: operation was cancelled [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 ┌ Warning: Verbosity logging macros are deprecated as they are not compatible with juliac-compiled programs │ caller = ip:0x0 └ @ Core :-1 ┌ Error: handle_connection handler error. │ │ =========================== │ HTTP Error message: │ │ ERROR: IOError: write: broken pipe (EPIPE) │ Stacktrace: │ [1] uv_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64) │ @ Base ./stream.jl:1102 │ [2] unsafe_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64) │ @ Base ./stream.jl:1156 │ [3] unsafe_write │ @ ~/.julia/packages/HTTP/4AUPl/src/Connections.jl:132 [inlined] │ [4] write │ @ ./strings/io.jl:246 [inlined] │ [5] write │ @ ./io.jl:794 [inlined] │ [6] unsafe_write(http::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}, p::Ptr{UInt8}, n::UInt64) │ @ HTTP.Streams ~/.julia/packages/HTTP/4AUPl/src/Streams.jl:95 │ [7] write │ @ ./strings/io.jl:246 [inlined] │ [8] write(io::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}, s::Base.CodeUnits{UInt8, String}) │ @ Base ./strings/basic.jl:803 │ [9] (::HTTP.Handlers.var"#streamhandler##0#streamhandler##1"{Main.var"##AWS S3 retries#283".var"#test_timeout##0#test_timeout##1"{Int64, String, Base.RefValue{Int64}}})(stream::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}) │ @ HTTP.Handlers ~/.julia/packages/HTTP/4AUPl/src/Handlers.jl:61 │ [10] #invokelatest#1 │ @ ./essentials.jl:1090 [inlined] │ [11] invokelatest │ @ ./essentials.jl:1086 [inlined] │ [12] handle_connection(f::Function, c::HTTP.Connections.Connection{Sockets.TCPSocket}, listener::HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, readtimeout::Int64, access_log::Nothing, verbose::Bool) │ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:469 │ [13] (::HTTP.Servers.var"#listenloop##0#listenloop##1"{HTTP.Handlers.var"#streamhandler##0#streamhandler##1"{Main.var"##AWS S3 retries#283".var"#test_timeout##0#test_timeout##1"{Int64, String, Base.RefValue{Int64}}}, HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, Set{HTTP.Connections.Connection}, Int64, Nothing, ReentrantLock, Bool, Base.Semaphore, HTTP.Connections.Connection{Sockets.TCPSocket}})() │ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:401 │ request = │ HTTP.Messages.Request: │ """ │ GET /mybucket/blob HTTP/1.1 │ host: 127.0.0.1:8081 │ x-amz-date: 20250305T225445Z │ x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 │ authorization: ****** │ accept: */* │ user-agent: object_store/0.11.2 │ │ """ └ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:483 [ Info: Server on 127.0.0.1:8081 closing ┌ Error: handle_connection handler error. │ │ =========================== │ HTTP Error message: │ │ ERROR: IOError: write: broken pipe (EPIPE) │ Stacktrace: │ [1] uv_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64) │ @ Base ./stream.jl:1102 │ [2] unsafe_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64) │ @ Base ./stream.jl:1156 │ [3] unsafe_write │ @ ~/.julia/packages/HTTP/4AUPl/src/Connections.jl:132 [inlined] │ [4] write │ @ ./strings/io.jl:246 [inlined] │ [5] write │ @ ./io.jl:794 [inlined] │ [6] unsafe_write(http::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}, p::Ptr{UInt8}, n::UInt64) │ @ HTTP.Streams ~/.julia/packages/HTTP/4AUPl/src/Streams.jl:95 │ [7] write │ @ ./strings/io.jl:246 [inlined] │ [8] write(io::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}, s::Base.CodeUnits{UInt8, String}) │ @ Base ./strings/basic.jl:803 │ [9] (::HTTP.Handlers.var"#streamhandler##0#streamhandler##1"{Main.var"##AWS S3 retries#283".var"#test_timeout##0#test_timeout##1"{Int64, String, Base.RefValue{Int64}}})(stream::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}) │ @ HTTP.Handlers ~/.julia/packages/HTTP/4AUPl/src/Handlers.jl:61 │ [10] #invokelatest#1 │ @ ./essentials.jl:1090 [inlined] │ [11] invokelatest │ @ ./essentials.jl:1086 [inlined] │ [12] handle_connection(f::Function, c::HTTP.Connections.Connection{Sockets.TCPSocket}, listener::HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, readtimeout::Int64, access_log::Nothing, verbose::Bool) │ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:469 │ [13] (::HTTP.Servers.var"#listenloop##0#listenloop##1"{HTTP.Handlers.var"#streamhandler##0#streamhandler##1"{Main.var"##AWS S3 retries#283".var"#test_timeout##0#test_timeout##1"{Int64, String, Base.RefValue{Int64}}}, HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, Set{HTTP.Connections.Connection}, Int64, Nothing, ReentrantLock, Bool, Base.Semaphore, HTTP.Connections.Connection{Sockets.TCPSocket}})() │ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:401 │ request = │ HTTP.Messages.Request: │ """ │ GET /mybucket/blob HTTP/1.1 │ host: 127.0.0.1:8081 │ x-amz-date: 20250305T225445Z │ x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 │ authorization: ****** │ accept: */* │ user-agent: object_store/0.11.2 │ │ """ └ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:483 [ Info: Listening on: 127.0.0.1:8081, thread id: 1 ┌ Error: handle_connection handler error. │ │ =========================== │ HTTP Error message: │ │ ERROR: IOError: write: broken pipe (EPIPE) │ Stacktrace: │ [1] uv_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64) │ @ Base ./stream.jl:1102 │ [2] unsafe_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64) │ @ Base ./stream.jl:1156 │ [3] unsafe_write │ @ ~/.julia/packages/HTTP/4AUPl/src/Connections.jl:132 [inlined] │ [4] write │ @ ./strings/io.jl:246 [inlined] │ [5] write │ @ ./io.jl:794 [inlined] │ [6] unsafe_write(http::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}, p::Ptr{UInt8}, n::UInt64) │ @ HTTP.Streams ~/.julia/packages/HTTP/4AUPl/src/Streams.jl:95 │ [7] write │ @ ./strings/io.jl:246 [inlined] │ [8] write(io::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}, s::Base.CodeUnits{UInt8, String}) │ @ Base ./strings/basic.jl:803 │ [9] (::HTTP.Handlers.var"#streamhandler##0#streamhandler##1"{Main.var"##AWS S3 retries#283".var"#test_timeout##0#test_timeout##1"{Int64, String, Base.RefValue{Int64}}})(stream::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}) │ @ HTTP.Handlers ~/.julia/packages/HTTP/4AUPl/src/Handlers.jl:61 │ [10] #invokelatest#1 │ @ ./essentials.jl:1090 [inlined] │ [11] invokelatest │ @ ./essentials.jl:1086 [inlined] │ [12] handle_connection(f::Function, c::HTTP.Connections.Connection{Sockets.TCPSocket}, listener::HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, readtimeout::Int64, access_log::Nothing, verbose::Bool) │ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:469 │ [13] (::HTTP.Servers.var"#listenloop##0#listenloop##1"{HTTP.Handlers.var"#streamhandler##0#streamhandler##1"{Main.var"##AWS S3 retries#283".var"#test_timeout##0#test_timeout##1"{Int64, String, Base.RefValue{Int64}}}, HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, Set{HTTP.Connections.Connection}, Int64, Nothing, ReentrantLock, Bool, Base.Semaphore, HTTP.Connections.Connection{Sockets.TCPSocket}})() │ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:401 │ request = │ HTTP.Messages.Request: │ """ │ PUT /mybucket/blob HTTP/1.1 │ content-length: 5 │ host: 127.0.0.1:8081 │ x-amz-date: 20250305T225449Z │ x-amz-content-sha256: 205830ca5b23bbe39ab510cfddc1dff2d9842e38b5fa7b7c48cd4ca7e44f92a1 │ authorization: ****** │ accept: */* │ user-agent: object_store/0.11.2 │ │ a,b,c""" └ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:483 ┌ Error: handle_connection handler error. │ │ =========================== │ HTTP Error message: │ │ ERROR: IOError: write: broken pipe (EPIPE) │ Stacktrace: │ [1] uv_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64) │ @ Base ./stream.jl:1102 │ [2] unsafe_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64) │ @ Base ./stream.jl:1156 │ [3] unsafe_write │ @ ~/.julia/packages/HTTP/4AUPl/src/Connections.jl:132 [inlined] │ [4] write │ @ ./strings/io.jl:246 [inlined] │ [5] write │ @ ./io.jl:794 [inlined] │ [6] unsafe_write(http::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}, p::Ptr{UInt8}, n::UInt64) │ @ HTTP.Streams ~/.julia/packages/HTTP/4AUPl/src/Streams.jl:95 │ [7] write │ @ ./strings/io.jl:246 [inlined] │ [8] write(io::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}, s::Base.CodeUnits{UInt8, String}) │ @ Base ./strings/basic.jl:803 │ [9] (::HTTP.Handlers.var"#streamhandler##0#streamhandler##1"{Main.var"##AWS S3 retries#283".var"#test_timeout##0#test_timeout##1"{Int64, String, Base.RefValue{Int64}}})(stream::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}) │ @ HTTP.Handlers ~/.julia/packages/HTTP/4AUPl/src/Handlers.jl:61 │ [10] #invokelatest#1 │ @ ./essentials.jl:1090 [inlined] │ [11] invokelatest │ @ ./essentials.jl:1086 [inlined] │ [12] handle_connection(f::Function, c::HTTP.Connections.Connection{Sockets.TCPSocket}, listener::HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, readtimeout::Int64, access_log::Nothing, verbose::Bool) │ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:469 │ [13] (::HTTP.Servers.var"#listenloop##0#listenloop##1"{HTTP.Handlers.var"#streamhandler##0#streamhandler##1"{Main.var"##AWS S3 retries#283".var"#test_timeout##0#test_timeout##1"{Int64, String, Base.RefValue{Int64}}}, HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, Set{HTTP.Connections.Connection}, Int64, Nothing, ReentrantLock, Bool, Base.Semaphore, HTTP.Connections.Connection{Sockets.TCPSocket}})() │ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:401 │ request = │ HTTP.Messages.Request: │ """ │ PUT /mybucket/blob HTTP/1.1 │ content-length: 5 │ host: 127.0.0.1:8081 │ x-amz-date: 20250305T225449Z │ x-amz-content-sha256: 205830ca5b23bbe39ab510cfddc1dff2d9842e38b5fa7b7c48cd4ca7e44f92a1 │ authorization: ****** │ accept: */* │ user-agent: object_store/0.11.2 │ │ a,b,c""" └ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:483 [ Info: Server on 127.0.0.1:8081 closing ┌ Error: handle_connection handler error. │ │ =========================== │ HTTP Error message: │ │ ERROR: IOError: write: broken pipe (EPIPE) │ Stacktrace: │ [1] uv_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64) │ @ Base ./stream.jl:1102 │ [2] unsafe_write(s::Sockets.TCPSocket, p::Ptr{UInt8}, n::UInt64) │ @ Base ./stream.jl:1156 │ [3] unsafe_write │ @ ~/.julia/packages/HTTP/4AUPl/src/Connections.jl:132 [inlined] │ [4] write │ @ ./strings/io.jl:246 [inlined] │ [5] write │ @ ./io.jl:794 [inlined] │ [6] unsafe_write(http::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}, p::Ptr{UInt8}, n::UInt64) │ @ HTTP.Streams ~/.julia/packages/HTTP/4AUPl/src/Streams.jl:95 │ [7] write │ @ ./strings/io.jl:246 [inlined] │ [8] write(io::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}, s::Base.CodeUnits{UInt8, String}) │ @ Base ./strings/basic.jl:803 │ [9] (::HTTP.Handlers.var"#streamhandler##0#streamhandler##1"{Main.var"##AWS S3 retries#283".var"#test_timeout##0#test_timeout##1"{Int64, String, Base.RefValue{Int64}}})(stream::HTTP.Streams.Stream{HTTP.Messages.Request, HTTP.Connections.Connection{Sockets.TCPSocket}}) │ @ HTTP.Handlers ~/.julia/packages/HTTP/4AUPl/src/Handlers.jl:61 │ [10] #invokelatest#1 │ @ ./essentials.jl:1090 [inlined] │ [11] invokelatest │ @ ./essentials.jl:1086 [inlined] │ [12] handle_connection(f::Function, c::HTTP.Connections.Connection{Sockets.TCPSocket}, listener::HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, readtimeout::Int64, access_log::Nothing, verbose::Bool) │ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:469 │ [13] (::HTTP.Servers.var"#listenloop##0#listenloop##1"{HTTP.Handlers.var"#streamhandler##0#streamhandler##1"{Main.var"##AWS S3 retries#283".var"#test_timeout##0#test_timeout##1"{Int64, String, Base.RefValue{Int64}}}, HTTP.Servers.Listener{Nothing, Sockets.TCPServer}, Set{HTTP.Connections.Connection}, Int64, Nothing, ReentrantLock, Bool, Base.Semaphore, HTTP.Connections.Connection{Sockets.TCPSocket}})() │ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:401 │ request = │ HTTP.Messages.Request: │ """ │ PUT /mybucket/blob HTTP/1.1 │ content-length: 5 │ host: 127.0.0.1:8081 │ x-amz-date: 20250305T225449Z │ x-amz-content-sha256: 205830ca5b23bbe39ab510cfddc1dff2d9842e38b5fa7b7c48cd4ca7e44f92a1 │ authorization: ****** │ accept: */* │ user-agent: object_store/0.11.2 │ │ a,b,c""" └ @ HTTP.Servers ~/.julia/packages/HTTP/4AUPl/src/Servers.jl:483 [ Info: Listening on: 127.0.0.1:8083, thread id: 1 [ Info: Server on 127.0.0.1:8083 closing [ Info: Listening on: 127.0.0.1:8083, thread id: 1 [ Info: Listening on: 127.0.0.1:8083, thread id: 1 [ Info: Server on 127.0.0.1:8083 closing [ Info: Server on 127.0.0.1:8083 closing [ Info: Listening on: 127.0.0.1:8081, thread id: 1 [ Info: Server on 127.0.0.1:8081 closing Error in testset "Timeout" on worker 179: Test Failed at /home/pkgeval/.julia/packages/RustyObjectStore/JF6aG/test/aws_s3_exception_tests.jl:667 Expression: nrequests == 1 + max_retries Evaluated: 2 == 3 Worker 179: 22:55:23 | maxrss 2.5% | mem 5.7% | START ( 4/20) test item "AzureConfig" at test/azure_api_tests.jl:1 Worker 179: 22:55:25 | maxrss 2.5% | mem 5.7% | DONE ( 4/20) test item "AzureConfig" 0.6 secs (67.1% compile), 72.49 K allocs (4.267 MB) Worker 179: 22:56:07 | maxrss 2.5% | mem 5.7% | START ( 5/20) test item "Basic BlobStorage exceptions" at test/azure_blobs_exception_tests.jl:1 Worker 179: 22:56:51 | maxrss 2.9% | mem 5.8% | DONE ( 5/20) test item "Basic BlobStorage exceptions" 42.3 secs (67.2% compile, <0.1% recompile, 1.6% GC), 8.21 M allocs (570.136 MB) Worker 179: 22:58:24 | maxrss 2.9% | mem 5.8% | START ( 6/20) test item "BlobStorage retries" at test/azure_blobs_exception_tests.jl:321 Worker 179: 22:59:02 | maxrss 2.9% | mem 5.9% | DONE ( 6/20) test item "BlobStorage retries" 36.9 secs (15.7% compile, <0.1% recompile, 0.3% GC), 4.72 M allocs (331.870 MB) Worker 179: 23:01:17 | maxrss 2.9% | mem 5.9% | START ( 7/20) test item "Basic BlobStorage usage" at test/basic_unified_tests.jl:686 Worker 179: 23:02:56 | maxrss 5.0% | mem 8.2% | DONE ( 7/20) test item "Basic BlobStorage usage" 96.0 secs (52.1% compile, <0.1% recompile, 3.3% GC), 11.00 M allocs (2.183 GB) Worker 179: 23:05:22 | maxrss 5.0% | mem 8.2% | START ( 8/20) test item "Basic BlobStorage usage (anonymous read enabled)" at test/basic_unified_tests.jl:719 Worker 179: 23:05:31 | maxrss 5.0% | mem 7.5% | DONE ( 8/20) test item "Basic BlobStorage usage (anonymous read enabled)" 7.3 secs (22.1% compile, <0.1% recompile, 12.9% GC), 1.36 M allocs (368.193 MB) Worker 179: 23:07:42 | maxrss 5.0% | mem 7.5% | START ( 9/20) test item "Basic AWS S3 usage" at test/basic_unified_tests.jl:744 Worker 179: 23:08:58 | maxrss 5.6% | mem 7.0% | DONE ( 9/20) test item "Basic AWS S3 usage" 74.6 secs (62.0% compile, <0.1% recompile, 4.4% GC), 7.84 M allocs (2.000 GB) Worker 179: 23:11:24 | maxrss 5.6% | mem 7.0% | START (10/20) test item "Basic AWS S3 usage (anonymous read enabled)" at test/basic_unified_tests.jl:778 Worker 179: 23:11:30 | maxrss 5.6% | mem 7.5% | DONE (10/20) test item "Basic AWS S3 usage (anonymous read enabled)" 4.3 secs (22.7% compile, 1.1% GC), 1.35 M allocs (368.008 MB) Worker 179: 23:13:40 | maxrss 5.6% | mem 7.5% | START (11/20) test item "Basic Snowflake Stage usage: AWS, non-encrypted" at test/basic_unified_tests.jl:805 Worker 179: 23:15:55 | maxrss 5.6% | mem 8.4% | DONE (11/20) test item "Basic Snowflake Stage usage: AWS, non-encrypted" 132.4 secs (56.3% compile, <0.1% recompile, 2.3% GC), 17.25 M allocs (2.272 GB) Worker 179: 23:18:23 | maxrss 5.6% | mem 8.4% | START (12/20) test item "Basic Snowflake Stage usage: AWS, encrypted" at test/basic_unified_tests.jl:822 Worker 179: 23:19:08 | maxrss 5.6% | mem 8.5% | DONE (12/20) test item "Basic Snowflake Stage usage: AWS, encrypted" 42.8 secs (21.8% compile, <0.1% recompile, 5.9% GC), 6.28 M allocs (1.423 GB) Worker 179: 2025-03-05T23:19:38.508555Z  WARN object_store_ffi::snowflake::client: Heartbeat failed: error sending request for url (http://127.0.0.1:8080/session/heartbeat) Worker 179: Caused by: Worker 179: 0: error sending request for url (http://127.0.0.1:8080/session/heartbeat) Worker 179: 1: client error (Connect) Worker 179: 2: tcp connect error: Connection refused (os error 111) Worker 179: 3: Connection refused (os error 111) Worker 179: 23:21:35 | maxrss 5.6% | mem 8.5% | START (13/20) test item "Basic Snowflake Stage usage: Azure, non-encrypted" at test/basic_unified_tests.jl:839 Worker 179: 23:22:21 | maxrss 5.6% | mem 9.0% | DONE (13/20) test item "Basic Snowflake Stage usage: Azure, non-encrypted" 44.5 secs (6.3% compile, 4.0% GC), 3.74 M allocs (1.269 GB) Worker 179: 2025-03-05T23:23:29.303590Z  WARN object_store_ffi::snowflake::client: Heartbeat failed: error sending request for url (http://127.0.0.1:8080/session/heartbeat) Worker 179: Caused by: Worker 179: 0: error sending request for url (http://127.0.0.1:8080/session/heartbeat) Worker 179: 1: client error (Connect) Worker 179: 2: tcp connect error: Connection refused (os error 111) Worker 179: 3: Connection refused (os error 111) Worker 179: 23:24:46 | maxrss 5.6% | mem 9.0% | START (14/20) test item "Basic Snowflake Stage usage: Azure, encrypted" at test/basic_unified_tests.jl:856 Worker 179: 23:25:43 | maxrss 6.4% | mem 10.0% | DONE (14/20) test item "Basic Snowflake Stage usage: Azure, encrypted" 54.4 secs (19.1% compile, 4.7% GC), 5.85 M allocs (1.399 GB) Worker 179: 23:26:15 | maxrss 6.4% | mem 10.0% | START (15/20) test item "destroy_* functions do not panic" at test/destroy_test.jl:1 Worker 179: 23:26:17 | maxrss 6.4% | mem 10.0% | DONE (15/20) test item "destroy_* functions do not panic" 0.3 secs (20.6% compile), 2.32 K allocs (123.304 KB) Worker 179: 23:26:21 | maxrss 6.4% | mem 10.0% | START (16/20) test item "Using object_store_ffi_jll" at test/release_test.jl:4 Worker 179: 23:26:23 | maxrss 6.4% | mem 10.0% | DONE (16/20) test item "Using object_store_ffi_jll" 0.2 secs (<0.1% compile), 1.51 K allocs (93.160 KB) Worker 179: 23:26:28 | maxrss 6.4% | mem 10.0% | START (17/20) test item "Handle object_store_ffi panic" at test/rust_panic_test.jl:1 Worker 179: 23:26:38 | maxrss 6.4% | mem 10.0% | DONE (17/20) test item "Handle object_store_ffi panic" 8.5 secs (6.1% compile, 0.4% recompile), 152.74 K allocs (9.089 MB) Worker 179: 2025-03-05T23:26:42.216264Z  WARN object_store_ffi::snowflake::client: Heartbeat failed: error sending request for url (http://127.0.0.1:8080/session/heartbeat) Worker 179: Caused by: Worker 179: 0: error sending request for url (http://127.0.0.1:8080/session/heartbeat) Worker 179: 1: client error (Connect) Worker 179: 2: tcp connect error: Connection refused (os error 111) Worker 179: 3: Connection refused (os error 111) Worker 179: 23:26:43 | maxrss 6.4% | mem 10.0% | START (18/20) test item "SnowflakeConfig" at test/snowflake_api_tests.jl:1 Worker 179: 23:26:45 | maxrss 6.4% | mem 10.0% | DONE (18/20) test item "SnowflakeConfig" 0.6 secs (63.7% compile), 114.00 K allocs (7.274 MB) Worker 179: 23:27:12 | maxrss 6.4% | mem 10.0% | START (19/20) test item "Basic Stage exceptions" at test/snowflake_stage_exception_tests.jl:1 Worker 179: 23:27:41 | maxrss 6.4% | mem 10.1% | DONE (19/20) test item "Basic Stage exceptions" 26.8 secs (15.3% compile), 1.27 M allocs (79.572 MB) Worker 179: 23:29:19 | maxrss 6.4% | mem 10.1% | START (20/20) test item "Snowflake Stage retries" at test/snowflake_stage_exception_tests.jl:158 Worker 179: 23:30:40 | maxrss 6.5% | mem 10.2% | DONE (20/20) test item "Snowflake Stage retries" 79.4 secs (11.9% compile, <0.1% recompile, 0.1% GC), 5.22 M allocs (331.857 MB) Worker 179: [179] signal 15: Terminated ====================================================================================== Information request received. A stacktrace will print followed by a 1.0 second profile ====================================================================================== cmd: /opt/julia/bin/julia 174 running 1 of 2 signal (10): User defined signal 1 unknown function (ip: 0x780714639f14) at /lib/x86_64-linux-gnu/libc.so.6 pthread_cond_wait at /lib/x86_64-linux-gnu/libc.so.6 (unknown line) uv_cond_wait at /workspace/srcdir/libuv/src/unix/thread.c:822 ijl_task_get_next at /source/src/scheduler.c:520 poptask at ./task.jl:1187 wait at ./task.jl:1199 task_done_hook at ./task.jl:839 jfptr_task_done_hook_74888.1 at /opt/julia/lib/julia/sys.so (unknown line) _jl_invoke at /source/src/gf.c:3475 [inlined] ijl_apply_generic at /source/src/gf.c:3675 jl_apply at /source/src/julia.h:2290 [inlined] jl_finish_task at /source/src/task.c:345 start_task at /source/src/task.c:1260 unknown function (ip: (nil)) at (unknown file) _ZNK4llvm6object7ELFFileINS0_7ELFTypeILNS_10endiannessE1ELb1EEEE25getSectionContentsAsArrayINS0_12Elf_Sym_ImplIS4_EEEENS_8ExpectedINS_8ArrayRefIT_EEEERKNS0_13Elf_Shdr_ImplIS4_EE at /opt/julia/bin/../lib/julia/libLLVM.so.19.1jl (unknown line) unknown function (ip: 0x7806fb2feda8) at /opt/julia/lib/julia/sys.so unknown function (ip: 0x1b0227f) at (unknown file) unknown function (ip: 0x7806d010b8b7) at (unknown file) unknown function (ip: (nil)) at (unknown file) ============================================================== Profile collected. A report will print at the next yield point ============================================================== ====================================================================================== Information request received. A stacktrace will print followed by a 1.0 second profile ====================================================================================== cmd: /opt/julia/bin/julia 1 running 0 of 2 signal (10): User defined signal 1 unknown function (ip: 0x7a49b03c9f14) at /lib/x86_64-linux-gnu/libc.so.6 pthread_cond_wait at /lib/x86_64-linux-gnu/libc.so.6 (unknown line) uv_cond_wait at /workspace/srcdir/libuv/src/unix/thread.c:822 ijl_task_get_next at /source/src/scheduler.c:520 poptask at ./task.jl:1187 wait at ./task.jl:1199 task_done_hook at ./task.jl:839 jfptr_task_done_hook_74888.1 at /opt/julia/lib/julia/sys.so (unknown line) _jl_invoke at /source/src/gf.c:3475 [inlined] ijl_apply_generic at /source/src/gf.c:3675 jl_apply at /source/src/julia.h:2290 [inlined] jl_finish_task at /source/src/task.c:345 start_task at /source/src/task.c:1260 unknown function (ip: (nil)) at (unknown file) epoll_pwait at /lib/x86_64-linux-gnu/libc.so.6 (unknown line) uv__io_poll at /workspace/srcdir/libuv/src/unix/linux.c:1404 uv_run at /workspace/srcdir/libuv/src/unix/core.c:430 ijl_task_get_next at /source/src/scheduler.c:454 poptask at ./task.jl:1187 wait at ./task.jl:1199 #wait#400 at ./condition.jl:141 wait at ./condition.jl:136 [inlined] wait at ./process.jl:694 wait at ./process.jl:687 jfptr_wait_79976.1 at /opt/julia/lib/julia/sys.so (unknown line) _jl_invoke at /source/src/gf.c:3475 [inlined] ijl_apply_generic at /source/src/gf.c:3675 subprocess_handler at /source/usr/share/julia/stdlib/v1.13/Pkg/src/Operations.jl:2404 unknown function (ip: 0x7a49a72a6da3) at (unknown file) _jl_invoke at /source/src/gf.c:3475 [inlined] ijl_apply_generic at /source/src/gf.c:3675 #203 at /source/usr/share/julia/stdlib/v1.13/Pkg/src/Operations.jl:2344 withenv at ./env.jl:265 #188 at /source/usr/share/julia/stdlib/v1.13/Pkg/src/Operations.jl:2165 with_temp_env at /source/usr/share/julia/stdlib/v1.13/Pkg/src/Operations.jl:2023 #184 at /source/usr/share/julia/stdlib/v1.13/Pkg/src/Operations.jl:2132 #mktempdir#21 at ./file.jl:899 unknown function (ip: 0x7a49a729401c) at (unknown file) _jl_invoke at /source/src/gf.c:3475 [inlined] ijl_apply_generic at /source/src/gf.c:3675 mktempdir at ./file.jl:895 mktempdir at ./file.jl:895 #sandbox#180 at /source/usr/share/julia/stdlib/v1.13/Pkg/src/Operations.jl:2079 [inlined] sandbox at /source/usr/share/julia/stdlib/v1.13/Pkg/src/Operations.jl:2071 unknown function (ip: 0x7a49a7292c09) at (unknown file) _jl_invoke at /source/src/gf.c:3475 [inlined] ijl_apply_generic at /source/src/gf.c:3675 #test#191 at /source/usr/share/julia/stdlib/v1.13/Pkg/src/Operations.jl:2329 test at /source/usr/share/julia/stdlib/v1.13/Pkg/src/Operations.jl:2241 [inlined] #test#170 at /source/usr/share/julia/stdlib/v1.13/Pkg/src/API.jl:486 test at /source/usr/share/julia/stdlib/v1.13/Pkg/src/API.jl:465 unknown function (ip: 0x7a49a7291e51) at (unknown file) _jl_invoke at /source/src/gf.c:3475 [inlined] ijl_apply_generic at /source/src/gf.c:3675 #test#84 at /source/usr/share/julia/stdlib/v1.13/Pkg/src/API.jl:164 unknown function (ip: 0x7a49a728f938) at (unknown file) _jl_invoke at /source/src/gf.c:3475 [inlined] ijl_apply_generic at /source/src/gf.c:3675 test at /source/usr/share/julia/stdlib/v1.13/Pkg/src/API.jl:153 #test#82 at /source/usr/share/julia/stdlib/v1.13/Pkg/src/API.jl:152 test at /source/usr/share/julia/stdlib/v1.13/Pkg/src/API.jl:152 [inlined] #test#81 at /source/usr/share/julia/stdlib/v1.13/Pkg/src/API.jl:151 [inlined] test at /source/usr/share/julia/stdlib/v1.13/Pkg/src/API.jl:151 unknown function (ip: 0x7a49a7286fef) at (unknown file) _jl_invoke at /source/src/gf.c:3475 [inlined] ijl_apply_generic at /source/src/gf.c:3675 jl_apply at /source/src/julia.h:2290 [inlined] do_call at /source/src/interpreter.c:124 eval_value at /source/src/interpreter.c:242 eval_stmt_value at /source/src/interpreter.c:193 [inlined] eval_body at /source/src/interpreter.c:692 eval_body at /source/src/interpreter.c:557 eval_body at /source/src/interpreter.c:557 jl_interpret_toplevel_thunk at /source/src/interpreter.c:897 jl_toplevel_eval_flex at /source/src/toplevel.c:1032 jl_toplevel_eval_flex at /source/src/toplevel.c:972 ijl_toplevel_eval at /source/src/toplevel.c:1044 ijl_toplevel_eval_in at /source/src/toplevel.c:1089 eval at ./boot.jl:488 include_string at ./loading.jl:2855 _jl_invoke at /source/src/gf.c:3475 [inlined] ijl_apply_generic at /source/src/gf.c:3675 _include at ./loading.jl:2915 include at ./Base.jl:306 exec_options at ./client.jl:328 _start at ./client.jl:560 jfptr__start_63990.1 at /opt/julia/lib/julia/sys.so (unknown line) _jl_invoke at /source/src/gf.c:3475 [inlined] ijl_apply_generic at /source/src/gf.c:3675 jl_apply at /source/src/julia.h:2290 [inlined] true_main at /source/src/jlapi.c:964 jl_repl_entrypoint at /source/src/jlapi.c:1124 main at /source/cli/loader_exe.c:58 unknown function (ip: 0x7a49b036b249) at /lib/x86_64-linux-gnu/libc.so.6 __libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line) unknown function (ip: 0x4010b8) at /workspace/srcdir/glibc-2.17/csu/../sysdeps/x86_64/start.S unknown function (ip: (nil)) at (unknown file) ============================================================== Profile collected. A report will print at the next yield point ============================================================== ┌ Warning: There were no samples collected in one or more groups. │ This may be due to idle threads, or you may need to run your │ program longer (perhaps by running it multiple times), │ or adjust the delay between samples with `Profile.init()`. └ @ Profile /opt/julia/share/julia/stdlib/v1.13/Profile/src/Profile.jl:1362 Overhead ╎ [+additional indent] Count File:Line Function ========================================================= Thread 1 (interactive) Task 0x00007a49a2200010 Total snapshots: 1. Utilization: 0% ╎1 @Base/client.jl:560 _start() ╎ 1 @Base/client.jl:328 exec_options(opts::Base.JLOptions) ╎ 1 @Base/Base.jl:306 include(mod::Module, _path::String) ╎ 1 @Base/loading.jl:2915 _include(mapexpr::Function, mod::Module, _path::S… ╎ 1 @Base/loading.jl:2855 include_string(mapexpr::typeof(identity), mod::M… ╎ 1 @Base/boot.jl:488 eval(m::Module, e::Any) ╎ ╎ 1 @Pkg/src/API.jl:151 kwcall(::@NamedTuple{julia_args::Cmd}, ::typeof(… ╎ ╎ 1 @Pkg/src/API.jl:151 #test#81 ╎ ╎ 1 @Pkg/src/API.jl:152 test ╎ ╎ 1 @Pkg/src/API.jl:152 test(pkgs::Vector{String}; kwargs::Base.Pairs… ╎ ╎ 1 @Pkg/src/API.jl:153 kwcall(::@NamedTuple{julia_args::Cmd}, ::typ… ╎ ╎ ╎ 1 @Pkg/src/API.jl:164 test(pkgs::Vector{Pkg.Types.PackageSpec}; i… ╎ ╎ ╎ 1 @Pkg/src/API.jl:465 kwcall(::@NamedTuple{julia_args::Cmd, io::… ╎ ╎ ╎ 1 @Pkg/src/API.jl:486 test(ctx::Pkg.Types.Context, pkgs::Vector… ╎ ╎ ╎ 1 @Pkg/…Operations.jl:2241 test ╎ ╎ ╎ 1 @Pkg/…perations.jl:2329 test(ctx::Pkg.Types.Context, pkgs::… ╎ ╎ ╎ ╎ 1 @Pkg/…perations.jl:2071 kwcall(::@NamedTuple{preferences::… ╎ ╎ ╎ ╎ 1 @Pkg/…perations.jl:2079 #sandbox#180 ╎ ╎ ╎ ╎ 1 @Base/file.jl:895 mktempdir(fn::Function) ╎ ╎ ╎ ╎ 1 @Base/file.jl:895 mktempdir(fn::Function, parent::Strin… ╎ ╎ ╎ ╎ 1 @Base/file.jl:899 mktempdir(fn::Pkg.Operations.var"#18… ╎ ╎ ╎ ╎ ╎ 1 @Pkg/…rations.jl:2132 (::Pkg.Operations.var"#184#185"… ╎ ╎ ╎ ╎ ╎ 1 @Pkg/…rations.jl:2023 with_temp_env(fn::Pkg.Operatio… ╎ ╎ ╎ ╎ ╎ 1 @Pkg/…ations.jl:2165 (::Pkg.Operations.var"#188#189… ╎ ╎ ╎ ╎ ╎ 1 @Base/env.jl:265 withenv(::Pkg.Operations.var"#203… ╎ ╎ ╎ ╎ ╎ 1 @Pkg/…tions.jl:2344 (::Pkg.Operations.var"#203#20… ╎ ╎ ╎ ╎ ╎ ╎ 1 @Pkg/…tions.jl:2404 subprocess_handler(cmd::Cmd,… ╎ ╎ ╎ ╎ ╎ ╎ 1 @Base/…cess.jl:687 wait(x::Base.Process) ╎ ╎ ╎ ╎ ╎ ╎ 1 @Base/…cess.jl:694 wait(x::Base.Process, syncd… ╎ ╎ ╎ ╎ ╎ ╎ 1 @Base/…ion.jl:136 wait ╎ ╎ ╎ ╎ ╎ ╎ 1 @Base/…ion.jl:141 wait(c::Base.GenericCondit… ╎ ╎ ╎ ╎ ╎ ╎ ╎ 1 @Base/…sk.jl:1199 wait() ╎ ╎ ╎ ╎ ╎ ╎ ╎ 1 @Base/…sk.jl:1187 poptask(W::Base.Intrusiv… Thread 2 (default) Task 0x00007a49a61fc3d0 Total snapshots: 1. Utilization: 0% ╎1 @Base/task.jl:839 task_done_hook(t::Task) ╎ 1 @Base/task.jl:1199 wait() ╎ 1 @Base/task.jl:1187 poptask(W::Base.IntrusiveLinkedListSynchronized{Task}) [174] signal 15: Terminated in expression starting at /home/pkgeval/.julia/packages/RustyObjectStore/JF6aG/test/runtests.jl:4 unknown function (ip: 0x780714639f14) at /lib/x86_64-linux-gnu/libc.so.6 pthread_cond_wait at /lib/x86_64-linux-gnu/libc.so.6 (unknown line) uv_cond_wait at /workspace/srcdir/libuv/src/unix/thread.c:822 ijl_task_get_next at /source/src/scheduler.c:520 poptask at ./task.jl:1187 wait at ./task.jl:1199 PkgEval terminated after 2724.0s: test duration exceeded the time limit