Package evaluation to test DistributedNext on Julia 1.14.0-DEV.3141 (fbd24f51d0*) started at 2026-09-08T14:02:31.190 ################################################################################ # Set-up # Installing PkgEval dependencies (TestEnv)... Activating project at `~/.julia/environments/v1.14` Set-up completed after 15.16s ################################################################################ # Installation # Installing DistributedNext... Resolving package versions... Updating `~/.julia/environments/v1.14/Project.toml` [fab6aee4] + DistributedNext v1.3.2 Updating `~/.julia/environments/v1.14/Manifest.toml` [fab6aee4] + DistributedNext v1.3.2 [076d061b] + HashArrayMappedTries v0.2.0 [aea7be01] + PrecompileTools v1.3.4 [21216c6a] + Preferences v1.5.2 [7e506255] + ScopedValues v1.6.2 [ade2ca70] + Dates v1.11.0 [56ddb016] + Logging v1.11.0 [de0858da] + Printf v1.11.0 [9a3f8284] + Random v1.11.0 [ea8e919c] + SHA v1.13.0 [9e88b42a] + Serialization v1.11.0 [6462fe0b] + Sockets v1.11.0 [fa267f1f] + TOML v1.0.3 [4ec0a83e] + Unicode v1.11.0 Installation completed after 1.36s ################################################################################ # Precompilation # Precompiling PkgEval dependencies... Precompiling package dependencies... Precompiling project... 18.1 s ✓ DistributedNext 8.5 s ✓ DistributedNext → ReviseExt 2 dependencies successfully precompiled in 27 seconds. 48 already precompiled. Precompilation completed after 42.47s ################################################################################ # Testing # Testing DistributedNext Status `/tmp/jl_d9yOUy/Project.toml` [4c88cf16] Aqua v0.8.16 [fab6aee4] DistributedNext v1.3.2 [aea7be01] PrecompileTools v1.3.4 [295af30f] Revise v3.17.0 [7e506255] ScopedValues v1.6.2 [8ba89e20] Distributed v1.12.0 [37e2e46d] LinearAlgebra v1.14.0 [44cfe95a] Pkg v1.14.0 [9a3f8284] Random v1.11.0 [9e88b42a] Serialization v1.11.0 [6462fe0b] Sockets v1.11.0 [8dfed614] Test v1.11.0 Status `/tmp/jl_d9yOUy/Manifest.toml` [4c88cf16] Aqua v0.8.16 [da1fd8a2] CodeTracking v3.0.2 [34da2185] Compat v4.18.1 [807dbc54] Compiler v0.1.1 [fab6aee4] DistributedNext v1.3.2 [076d061b] HashArrayMappedTries v0.2.0 [aa1ae85d] JuliaInterpreter v0.11.4 [6f1432cf] LoweredCodeUtils v3.8.0 [bac558e1] OrderedCollections v2.0.1 [aea7be01] PrecompileTools v1.3.4 [21216c6a] Preferences v1.5.2 [295af30f] Revise v3.17.0 [7e506255] ScopedValues v1.6.2 [0dad84c5] ArgTools v1.2.0 [56f22d72] Artifacts v1.11.0 [2a0f44e3] Base64 v1.11.0 [8bf52ea8] CRC32c v1.11.0 [ade2ca70] Dates v1.11.0 [8ba89e20] Distributed v1.12.0 [f43a241f] Downloads v1.7.0 [7b1f6079] FileWatching v1.11.0 [b77e0a4c] InteractiveUtils v1.11.0 [ac6e5ff7] JuliaSyntaxHighlighting v1.13.0 [b27032c2] LibCURL v1.0.0 [76f85450] LibGit2 v1.11.0 [8f399da3] Libdl v1.11.0 [37e2e46d] LinearAlgebra v1.14.0 [56ddb016] Logging v1.11.0 [d6f4376e] Markdown v1.11.0 [ca575930] NetworkOptions v1.3.0 [44cfe95a] Pkg v1.14.0 [de0858da] Printf v1.11.0 [3fa0cd96] REPL v1.11.0 [9a3f8284] Random v1.11.0 [ea8e919c] SHA v1.13.0 [9e88b42a] Serialization v1.11.0 [6462fe0b] Sockets v1.11.0 [f489334b] StyledStrings v1.13.0 [fa267f1f] TOML v1.0.3 [a4e569a6] Tar v1.10.0 [8dfed614] Test v1.11.0 [cf7118a7] UUIDs v1.11.0 [4ec0a83e] Unicode v1.11.0 [e66e0078] CompilerSupportLibraries_jll v1.5.7+0 [deac9b47] LibCURL_jll v8.22.0+0 [e37daf67] LibGit2_jll v1.9.7+0 [29816b5a] LibSSH2_jll v1.11.104+0 [14a3606d] MozillaCACerts_jll v2026.8.13 [4536629a] OpenBLAS_jll v0.3.34+0 [458c3c95] OpenSSL_jll v3.5.8+0 [efcefdf7] PCRE2_jll v10.48.0+0 [83775a58] Zlib_jll v1.3.2+0 [3161d3a3] Zstd_jll v1.5.7+1 [8e850b90] libblastrampoline_jll v5.15.0+0 [8e850ede] nghttp2_jll v1.70.0+0 [3f19e933] p7zip_jll v17.8.2+0 Testing Running tests... Resolving package versions... Updating `/tmp/jl_d9yOUy/Project.toml` [00483490] + LibSSH v1.2.0 Updating `/tmp/jl_d9yOUy/Manifest.toml` [fa961155] + CEnum v0.5.0 [ffbed154] + DocStringExtensions v0.9.5 [692b3bcd] + JLLWrappers v1.8.0 [00483490] + LibSSH v1.2.0 [b39eb1a6] + Kerberos_krb5_jll v1.21.3+0 [a8d4f100] + libssh_jll v0.12.2+0 [pid 33] Waiting for background task / IO / timer to finish: Handle type uv_handle_t->data timer 0x13396dd0->0x7f843fe4c7c0 timer 0x136c6c70->0x7f84412d1570 This means that a package has started a background task or event source that has not finished running. For precompilation to complete successfully, the event source needs to be closed explicitly. See the developer documentation on fixing precompilation hangs for more help. 1 dependency had output during precompilation: ┌ LibSSH │ [Output was shown above] └ Testing SSH addprocs with 9 workers... ┌ Error: Exception launching on machine 128.30.92.239 │ exception = │ IOError: could not spawn `ssh -T -a -x -o ClearAllForwardings=yes -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR -p 2222 128.30.92.239 "sh -l -c 'cd -- /home/pkgeval/.julia/packages/DistributedNext/42dx6/test │ export JULIA_PKGEVAL=true │ export JULIA_LOAD_PATH='\\''@:/tmp/jl_d9yOUy'\\'' │ export JULIA_PKG_PRECOMPILE_AUTO=0 │ export JULIA_CPU_THREADS=1 │ export JULIA_NUM_THREADS=1 │ export JULIA_DEPOT_PATH='\\''/home/pkgeval/.julia:/usr/local/share/julia:'\\'' │ export JULIA_NUM_PRECOMPILE_TASKS=1 │ exec /opt/julia/bin/julia '\\''--project=/tmp/jl_d9yOUy/Project.toml'\\'' -C native -J/opt/julia/lib/julia/sys.so --pkgimages=existing -g1 --startup-file=no --check-bounds=yes --startup-file=no --depwarn=error -E '\\''using DistributedNext; DistributedNext.start_worker()'\\'''"`: no such file or directory (ENOENT) │ Stacktrace: │ [1] _spawn_primitive(file::String, cmd::Cmd, stdio::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:156 │ [2] _spawn(cmd::Cmd, stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:177 [inlined] │ [3] (::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken})(stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}) │ @ Base process.jl:169 [inlined] │ [4] setup_stdios(f::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken}, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}) │ @ Base process.jl:267 │ [5] _spawn(cmd::Cmd, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:168 [inlined] │ [6] open(cmds::Cmd, stdio::Base.DevNull; write::Bool, read::Bool, cancel::Base.UseDefaultToken) │ @ Base process.jl:438 │ [7] open(cmds::Cmd, mode::String, stdio::Base.DevNull; cancel::Base.UseDefaultToken) │ @ Base process.jl:410 │ [8] open(cmds::Cmd, mode::String, stdio::Base.DevNull) │ @ Base process.jl:407 [inlined] │ [9] launch_on_machine(manager::DistributedNext.SSHManager, machine::String, cnt::Int64, params::Dict{Symbol, Any}, launched::Vector{WorkerConfig}, launch_ntfy::Condition) │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:379 │ [10] (::DistributedNext.var"#launch##0#launch##1"{DistributedNext.SSHManager, Dict{Symbol, Any}, Vector{WorkerConfig}, Condition, String, String, Int64})() │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:195 └ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:197 ┌ Error: Exception launching on machine 127.0.0.1 │ exception = │ IOError: could not spawn `ssh -T -a -x -o ClearAllForwardings=yes -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR -p 2222 127.0.0.1 "sh -l -c 'cd -- /home/pkgeval/.julia/packages/DistributedNext/42dx6/test │ export JULIA_PKGEVAL=true │ export JULIA_LOAD_PATH='\\''@:/tmp/jl_d9yOUy'\\'' │ export JULIA_PKG_PRECOMPILE_AUTO=0 │ export JULIA_CPU_THREADS=1 │ export JULIA_NUM_THREADS=1 │ export JULIA_DEPOT_PATH='\\''/home/pkgeval/.julia:/usr/local/share/julia:'\\'' │ export JULIA_NUM_PRECOMPILE_TASKS=1 │ exec /opt/julia/bin/julia '\\''--project=/tmp/jl_d9yOUy/Project.toml'\\'' -C native -J/opt/julia/lib/julia/sys.so --pkgimages=existing -g1 --startup-file=no --check-bounds=yes --startup-file=no --depwarn=error -E '\\''using DistributedNext; DistributedNext.start_worker()'\\'''"`: no such file or directory (ENOENT) │ Stacktrace: │ [1] _spawn_primitive(file::String, cmd::Cmd, stdio::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:156 │ [2] _spawn(cmd::Cmd, stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:177 [inlined] │ [3] (::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken})(stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}) │ @ Base process.jl:169 [inlined] │ [4] setup_stdios(f::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken}, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}) │ @ Base process.jl:267 │ [5] _spawn(cmd::Cmd, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:168 [inlined] │ [6] open(cmds::Cmd, stdio::Base.DevNull; write::Bool, read::Bool, cancel::Base.UseDefaultToken) │ @ Base process.jl:438 │ [7] open(cmds::Cmd, mode::String, stdio::Base.DevNull; cancel::Base.UseDefaultToken) │ @ Base process.jl:410 │ [8] open(cmds::Cmd, mode::String, stdio::Base.DevNull) │ @ Base process.jl:407 [inlined] │ [9] launch_on_machine(manager::DistributedNext.SSHManager, machine::String, cnt::Int64, params::Dict{Symbol, Any}, launched::Vector{WorkerConfig}, launch_ntfy::Condition) │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:379 │ [10] (::DistributedNext.var"#launch##0#launch##1"{DistributedNext.SSHManager, Dict{Symbol, Any}, Vector{WorkerConfig}, Condition, String, String, Int64})() │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:195 └ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:197 ┌ Error: Exception launching on machine localhost │ exception = │ IOError: could not spawn `ssh -T -a -x -o ClearAllForwardings=yes -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR -p 2222 localhost "sh -l -c 'cd -- /home/pkgeval/.julia/packages/DistributedNext/42dx6/test │ export JULIA_PKGEVAL=true │ export JULIA_LOAD_PATH='\\''@:/tmp/jl_d9yOUy'\\'' │ export JULIA_PKG_PRECOMPILE_AUTO=0 │ export JULIA_CPU_THREADS=1 │ export JULIA_NUM_THREADS=1 │ export JULIA_DEPOT_PATH='\\''/home/pkgeval/.julia:/usr/local/share/julia:'\\'' │ export JULIA_NUM_PRECOMPILE_TASKS=1 │ exec /opt/julia/bin/julia '\\''--project=/tmp/jl_d9yOUy/Project.toml'\\'' -C native -J/opt/julia/lib/julia/sys.so --pkgimages=existing -g1 --startup-file=no --check-bounds=yes --startup-file=no --depwarn=error -E '\\''using DistributedNext; DistributedNext.start_worker()'\\'''"`: no such file or directory (ENOENT) │ Stacktrace: │ [1] _spawn_primitive(file::String, cmd::Cmd, stdio::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:156 │ [2] _spawn(cmd::Cmd, stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:177 [inlined] │ [3] (::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken})(stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}) │ @ Base process.jl:169 [inlined] │ [4] setup_stdios(f::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken}, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}) │ @ Base process.jl:267 │ [5] _spawn(cmd::Cmd, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:168 [inlined] │ [6] open(cmds::Cmd, stdio::Base.DevNull; write::Bool, read::Bool, cancel::Base.UseDefaultToken) │ @ Base process.jl:438 │ [7] open(cmds::Cmd, mode::String, stdio::Base.DevNull; cancel::Base.UseDefaultToken) │ @ Base process.jl:410 │ [8] open(cmds::Cmd, mode::String, stdio::Base.DevNull) │ @ Base process.jl:407 [inlined] │ [9] launch_on_machine(manager::DistributedNext.SSHManager, machine::String, cnt::Int64, params::Dict{Symbol, Any}, launched::Vector{WorkerConfig}, launch_ntfy::Condition) │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:379 │ [10] (::DistributedNext.var"#launch##0#launch##1"{DistributedNext.SSHManager, Dict{Symbol, Any}, Vector{WorkerConfig}, Condition, String, String, Int64})() │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:195 └ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:197 SSHManager: Test Failed at /home/pkgeval/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:52 Expression: length(new_pids) == length(hosts) Evaluated: 0 == 9 Stacktrace: [1] macro expansion @ /opt/julia/share/julia/stdlib/v1.14/Test/src/Test.jl:789 [inlined] [2] (::var"#15#16"{UInt16})() @ Main ~/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:52 [3] (::LibSSH.var"#280#281"{var"#15#16"{UInt16}, Vector{Any}, Timer})() @ LibSSH ~/.julia/packages/LibSSH/aHL74/src/server.jl:875 Mixed ssh addprocs with :auto ┌ Error: Exception launching on machine 127.0.0.1 │ exception = │ IOError: could not spawn `ssh -T -a -x -o ClearAllForwardings=yes -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR -p 2222 127.0.0.1 "sh -l -c 'cd -- /home/pkgeval/.julia/packages/DistributedNext/42dx6/test │ export JULIA_PKGEVAL=true │ export JULIA_LOAD_PATH='\\''@:/tmp/jl_d9yOUy'\\'' │ export JULIA_PKG_PRECOMPILE_AUTO=0 │ export JULIA_CPU_THREADS=1 │ export JULIA_NUM_THREADS=1 │ export JULIA_DEPOT_PATH='\\''/home/pkgeval/.julia:/usr/local/share/julia:'\\'' │ export JULIA_NUM_PRECOMPILE_TASKS=1 │ exec /opt/julia/bin/julia '\\''--project=/tmp/jl_d9yOUy/Project.toml'\\'' -C native -J/opt/julia/lib/julia/sys.so --pkgimages=existing -g1 --startup-file=no --check-bounds=yes --startup-file=no --depwarn=error -E '\\''using DistributedNext; DistributedNext.start_worker()'\\'''"`: no such file or directory (ENOENT) │ Stacktrace: │ [1] _spawn_primitive(file::String, cmd::Cmd, stdio::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:156 │ [2] _spawn(cmd::Cmd, stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:177 [inlined] │ [3] (::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken})(stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}) │ @ Base process.jl:169 [inlined] │ [4] setup_stdios(f::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken}, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}) │ @ Base process.jl:267 │ [5] _spawn(cmd::Cmd, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:168 [inlined] │ [6] open(cmds::Cmd, stdio::Base.DevNull; write::Bool, read::Bool, cancel::Base.UseDefaultToken) │ @ Base process.jl:438 │ [7] open(cmds::Cmd, mode::String, stdio::Base.DevNull; cancel::Base.UseDefaultToken) │ @ Base process.jl:410 │ [8] open(cmds::Cmd, mode::String, stdio::Base.DevNull) │ @ Base process.jl:407 [inlined] │ [9] launch_on_machine(manager::DistributedNext.SSHManager, machine::String, cnt::Symbol, params::Dict{Symbol, Any}, launched::Vector{WorkerConfig}, launch_ntfy::Condition) │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:379 │ [10] (::DistributedNext.var"#launch##0#launch##1"{DistributedNext.SSHManager, Dict{Symbol, Any}, Vector{WorkerConfig}, Condition, String, String, Symbol})() │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:195 └ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:197 ┌ Error: Exception launching on machine localhost │ exception = │ IOError: could not spawn `ssh -T -a -x -o ClearAllForwardings=yes -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR -p 2222 localhost "sh -l -c 'cd -- /home/pkgeval/.julia/packages/DistributedNext/42dx6/test │ export JULIA_PKGEVAL=true │ export JULIA_LOAD_PATH='\\''@:/tmp/jl_d9yOUy'\\'' │ export JULIA_PKG_PRECOMPILE_AUTO=0 │ export JULIA_CPU_THREADS=1 │ export JULIA_NUM_THREADS=1 │ export JULIA_DEPOT_PATH='\\''/home/pkgeval/.julia:/usr/local/share/julia:'\\'' │ export JULIA_NUM_PRECOMPILE_TASKS=1 │ exec /opt/julia/bin/julia '\\''--project=/tmp/jl_d9yOUy/Project.toml'\\'' -C native -J/opt/julia/lib/julia/sys.so --pkgimages=existing -g1 --startup-file=no --check-bounds=yes --startup-file=no --depwarn=error -E '\\''using DistributedNext; DistributedNext.start_worker()'\\'''"`: no such file or directory (ENOENT) │ Stacktrace: │ [1] _spawn_primitive(file::String, cmd::Cmd, stdio::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:156 │ [2] _spawn(cmd::Cmd, stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:177 [inlined] │ [3] (::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken})(stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}) │ @ Base process.jl:169 [inlined] │ [4] setup_stdios(f::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken}, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}) │ @ Base process.jl:267 │ [5] _spawn(cmd::Cmd, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:168 [inlined] │ [6] open(cmds::Cmd, stdio::Base.DevNull; write::Bool, read::Bool, cancel::Base.UseDefaultToken) │ @ Base process.jl:438 │ [7] open(cmds::Cmd, mode::String, stdio::Base.DevNull; cancel::Base.UseDefaultToken) │ @ Base process.jl:410 │ [8] open(cmds::Cmd, mode::String, stdio::Base.DevNull) │ @ Base process.jl:407 [inlined] │ [9] launch_on_machine(manager::DistributedNext.SSHManager, machine::String, cnt::Int64, params::Dict{Symbol, Any}, launched::Vector{WorkerConfig}, launch_ntfy::Condition) │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:379 │ [10] (::DistributedNext.var"#launch##0#launch##1"{DistributedNext.SSHManager, Dict{Symbol, Any}, Vector{WorkerConfig}, Condition, String, String, Int64})() │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:195 └ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:197 SSHManager: Test Failed at /home/pkgeval/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:57 Expression: length(new_pids) == 2 + Sys.CPU_THREADS Evaluated: 0 == 3 Stacktrace: [1] macro expansion @ /opt/julia/share/julia/stdlib/v1.14/Test/src/Test.jl:789 [inlined] [2] (::var"#15#16"{UInt16})() @ Main ~/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:57 [3] (::LibSSH.var"#280#281"{var"#15#16"{UInt16}, Vector{Any}, Timer})() @ LibSSH ~/.julia/packages/LibSSH/aHL74/src/server.jl:875 Mixed ssh addprocs with numeric counts ┌ Error: Exception launching on machine 127.0.0.1 │ exception = │ IOError: could not spawn `ssh -T -a -x -o ClearAllForwardings=yes -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR -p 2222 127.0.0.1 "sh -l -c 'cd -- /home/pkgeval/.julia/packages/DistributedNext/42dx6/test │ export JULIA_PKGEVAL=true │ export JULIA_LOAD_PATH='\\''@:/tmp/jl_d9yOUy'\\'' │ export JULIA_PKG_PRECOMPILE_AUTO=0 │ export JULIA_CPU_THREADS=1 │ export JULIA_NUM_THREADS=1 │ export JULIA_DEPOT_PATH='\\''/home/pkgeval/.julia:/usr/local/share/julia:'\\'' │ export JULIA_NUM_PRECOMPILE_TASKS=1 │ exec /opt/julia/bin/julia '\\''--project=/tmp/jl_d9yOUy/Project.toml'\\'' -C native -J/opt/julia/lib/julia/sys.so --pkgimages=existing -g1 --startup-file=no --check-bounds=yes --startup-file=no --depwarn=error -E '\\''using DistributedNext; DistributedNext.start_worker()'\\'''"`: no such file or directory (ENOENT) │ Stacktrace: │ [1] _spawn_primitive(file::String, cmd::Cmd, stdio::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:156 │ [2] _spawn(cmd::Cmd, stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:177 [inlined] │ [3] (::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken})(stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}) │ @ Base process.jl:169 [inlined] │ [4] setup_stdios(f::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken}, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}) │ @ Base process.jl:267 │ [5] _spawn(cmd::Cmd, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:168 [inlined] │ [6] open(cmds::Cmd, stdio::Base.DevNull; write::Bool, read::Bool, cancel::Base.UseDefaultToken) │ @ Base process.jl:438 │ [7] open(cmds::Cmd, mode::String, stdio::Base.DevNull; cancel::Base.UseDefaultToken) │ @ Base process.jl:410 │ [8] open(cmds::Cmd, mode::String, stdio::Base.DevNull) │ @ Base process.jl:407 [inlined] │ [9] launch_on_machine(manager::DistributedNext.SSHManager, machine::String, cnt::Int64, params::Dict{Symbol, Any}, launched::Vector{WorkerConfig}, launch_ntfy::Condition) │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:379 │ [10] (::DistributedNext.var"#launch##0#launch##1"{DistributedNext.SSHManager, Dict{Symbol, Any}, Vector{WorkerConfig}, Condition, String, String, Int64})() │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:195 └ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:197 ┌ Error: Exception launching on machine localhost │ exception = │ IOError: could not spawn `ssh -T -a -x -o ClearAllForwardings=yes -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR -p 2222 localhost "sh -l -c 'cd -- /home/pkgeval/.julia/packages/DistributedNext/42dx6/test │ export JULIA_PKGEVAL=true │ export JULIA_LOAD_PATH='\\''@:/tmp/jl_d9yOUy'\\'' │ export JULIA_PKG_PRECOMPILE_AUTO=0 │ export JULIA_CPU_THREADS=1 │ export JULIA_NUM_THREADS=1 │ export JULIA_DEPOT_PATH='\\''/home/pkgeval/.julia:/usr/local/share/julia:'\\'' │ export JULIA_NUM_PRECOMPILE_TASKS=1 │ exec /opt/julia/bin/julia '\\''--project=/tmp/jl_d9yOUy/Project.toml'\\'' -C native -J/opt/julia/lib/julia/sys.so --pkgimages=existing -g1 --startup-file=no --check-bounds=yes --startup-file=no --depwarn=error -E '\\''using DistributedNext; DistributedNext.start_worker()'\\'''"`: no such file or directory (ENOENT) │ Stacktrace: │ [1] _spawn_primitive(file::String, cmd::Cmd, stdio::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:156 │ [2] _spawn(cmd::Cmd, stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:177 [inlined] │ [3] (::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken})(stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}) │ @ Base process.jl:169 [inlined] │ [4] setup_stdios(f::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken}, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}) │ @ Base process.jl:267 │ [5] _spawn(cmd::Cmd, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:168 [inlined] │ [6] open(cmds::Cmd, stdio::Base.DevNull; write::Bool, read::Bool, cancel::Base.UseDefaultToken) │ @ Base process.jl:438 │ [7] open(cmds::Cmd, mode::String, stdio::Base.DevNull; cancel::Base.UseDefaultToken) │ @ Base process.jl:410 │ [8] open(cmds::Cmd, mode::String, stdio::Base.DevNull) │ @ Base process.jl:407 [inlined] │ [9] launch_on_machine(manager::DistributedNext.SSHManager, machine::String, cnt::Int64, params::Dict{Symbol, Any}, launched::Vector{WorkerConfig}, launch_ntfy::Condition) │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:379 │ [10] (::DistributedNext.var"#launch##0#launch##1"{DistributedNext.SSHManager, Dict{Symbol, Any}, Vector{WorkerConfig}, Condition, String, String, Int64})() │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:195 └ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:197 SSHManager: Test Failed at /home/pkgeval/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:62 Expression: length(new_pids) == 5 Evaluated: 0 == 5 Stacktrace: [1] macro expansion @ /opt/julia/share/julia/stdlib/v1.14/Test/src/Test.jl:789 [inlined] [2] (::var"#15#16"{UInt16})() @ Main ~/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:62 [3] (::LibSSH.var"#280#281"{var"#15#16"{UInt16}, Vector{Any}, Timer})() @ LibSSH ~/.julia/packages/LibSSH/aHL74/src/server.jl:875 ssh addprocs with a port hint ┌ Error: Exception launching on machine localhost 127.0.0.1:[8000] │ exception = │ IOError: could not spawn `ssh -T -a -x -o ClearAllForwardings=yes -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o LogLevel=ERROR -p 2222 localhost "sh -l -c 'cd -- /home/pkgeval/.julia/packages/DistributedNext/42dx6/test │ export JULIA_PKGEVAL=true │ export JULIA_LOAD_PATH='\\''@:/tmp/jl_d9yOUy'\\'' │ export JULIA_PKG_PRECOMPILE_AUTO=0 │ export JULIA_CPU_THREADS=1 │ export JULIA_NUM_THREADS=1 │ export JULIA_DEPOT_PATH='\\''/home/pkgeval/.julia:/usr/local/share/julia:'\\'' │ export JULIA_NUM_PRECOMPILE_TASKS=1 │ exec /opt/julia/bin/julia --bind-to '\\''127.0.0.1:[8000]'\\'' --project=/tmp/jl_d9yOUy/Project.toml -C native -J/opt/julia/lib/julia/sys.so --pkgimages=existing -g1 --startup-file=no --check-bounds=yes --startup-file=no --depwarn=error -E '\\''using DistributedNext; DistributedNext.start_worker()'\\'''"`: no such file or directory (ENOENT) │ Stacktrace: │ [1] _spawn_primitive(file::String, cmd::Cmd, stdio::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:156 │ [2] _spawn(cmd::Cmd, stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:177 [inlined] │ [3] (::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken})(stdios::Memory{Union{RawFD, Base.IOServer, Base.SyncCloseFD, IO}}) │ @ Base process.jl:169 [inlined] │ [4] setup_stdios(f::Base.var"#_spawn##0#_spawn##1"{Cmd, Base.CancellationToken}, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}) │ @ Base process.jl:267 │ [5] _spawn(cmd::Cmd, stdios::Vector{Union{RawFD, Base.FileRedirect, Base.IOServer, IO}}, tok::Base.CancellationToken) │ @ Base process.jl:168 [inlined] │ [6] open(cmds::Cmd, stdio::Base.DevNull; write::Bool, read::Bool, cancel::Base.UseDefaultToken) │ @ Base process.jl:438 │ [7] open(cmds::Cmd, mode::String, stdio::Base.DevNull; cancel::Base.UseDefaultToken) │ @ Base process.jl:410 │ [8] open(cmds::Cmd, mode::String, stdio::Base.DevNull) │ @ Base process.jl:407 [inlined] │ [9] launch_on_machine(manager::DistributedNext.SSHManager, machine::String, cnt::Int64, params::Dict{Symbol, Any}, launched::Vector{WorkerConfig}, launch_ntfy::Condition) │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:379 │ [10] (::DistributedNext.var"#launch##0#launch##1"{DistributedNext.SSHManager, Dict{Symbol, Any}, Vector{WorkerConfig}, Condition, String, String, Int64})() │ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:195 └ @ DistributedNext ~/.julia/packages/DistributedNext/42dx6/src/managers.jl:197 UNHANDLED TASK ERROR: ArgumentError: Collection is empty, must contain exactly 1 element Stacktrace: [1] _only(x::Vector{Int64}, ::typeof(iterate)) @ Base.Iterators iterators.jl:1642 [inlined] [2] only(x::Vector{Int64}) @ Base.Iterators iterators.jl:1637 [3] (::var"#15#16"{UInt16})() @ Main ~/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:67 [4] (::LibSSH.var"#280#281"{var"#15#16"{UInt16}, Vector{Any}, Timer})() @ LibSSH ~/.julia/packages/LibSSH/aHL74/src/server.jl:875 SSHManager: Error During Test at /home/pkgeval/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:29 Got exception outside of a @test TaskFailedException Stacktrace: [1] wait(t::Task, tok::Base.CancellationToken; throw::Bool) @ Base task.jl:500 [2] wait(t::Task; throw::Bool, cancel::Base.UseDefaultToken) @ Base task.jl:494 [3] DemoServer(f::var"#15#16"{UInt16}, args::Int64; timeout::Int64, kill_timeout::Int64, kwargs::@Kwargs{auth_methods::Vector{LibSSH.AuthMethod}, allow_auth_none::Bool, verbose::Bool}) @ LibSSH ~/.julia/packages/LibSSH/aHL74/src/server.jl:907 [4] top-level scope @ ~/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:30 [5] macro expansion @ /opt/julia/share/julia/stdlib/v1.14/Test/src/Test.jl:2252 [inlined] [6] macro expansion @ ~/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:33 [inlined] nested task error: ArgumentError: Collection is empty, must contain exactly 1 element Stacktrace: [1] _only(x::Vector{Int64}, ::typeof(iterate)) @ Base.Iterators iterators.jl:1642 [inlined] [2] only(x::Vector{Int64}) @ Base.Iterators iterators.jl:1637 [3] (::var"#15#16"{UInt16})() @ Main ~/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:67 [4] (::LibSSH.var"#280#281"{var"#15#16"{UInt16}, Vector{Any}, Timer})() @ LibSSH ~/.julia/packages/LibSSH/aHL74/src/server.jl:875 Test Summary: | Fail Error Total Time SSHManager | 3 1 4 38.9s RNG of the outermost testset: Random.Xoshiro(0x6692730f4e173acc, 0xd5cea27e880f5779, 0x8f794f73ce7f3acf, 0xd7fabde43cd5ce54, 0x9d38190b989a0b8e) ERROR: LoadError: Some tests did not pass: 0 passed, 3 failed, 1 errored, 0 broken. in expression starting at /home/pkgeval/.julia/packages/DistributedNext/42dx6/test/sshmanager.jl:29 in expression starting at /home/pkgeval/.julia/packages/DistributedNext/42dx6/test/runtests.jl:15 Testing failed after 103.01s ERROR: LoadError: Package DistributedNext errored during testing Stacktrace: [1] pkgerror(msg::String) @ Pkg.Types /opt/julia/share/julia/stdlib/v1.14/Pkg/src/Types.jl:68 [2] test(ctx::Pkg.Types.Context, pkgs::Vector{PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool) @ Pkg.Operations /opt/julia/share/julia/stdlib/v1.14/Pkg/src/Operations.jl:3298 [3] test(ctx::Pkg.Types.Context, pkgs::Vector{PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::@Kwargs{io::IOContext{IO}}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.14/Pkg/src/API.jl:587 [4] test(pkgs::Vector{PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{julia_args::Cmd}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.14/Pkg/src/API.jl:172 [5] test(pkgs::Vector{String}; kwargs::@Kwargs{julia_args::Cmd}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.14/Pkg/src/API.jl:160 [6] test(pkg::String; kwargs::@Kwargs{julia_args::Cmd}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.14/Pkg/src/API.jl:159 [inlined] [7] top-level scope @ /PkgEval.jl/scripts/evaluate.jl:223 in expression starting at /PkgEval.jl/scripts/evaluate.jl:214 PkgEval failed after 191.37s: package tests unexpectedly errored