Package evaluation of BenchmarkCI on Julia 1.12.0-rc1.2 (995ff9db19*) started at 2025-07-14T17:11:37.888 ################################################################################ # Set-up # Installing PkgEval dependencies (TestEnv)... Set-up completed after 8.61s ################################################################################ # Installation # Installing BenchmarkCI... Resolving package versions... Updating `~/.julia/environments/v1.12/Project.toml` [20533458] + BenchmarkCI v0.1.9 Updating `~/.julia/environments/v1.12/Manifest.toml` [1520ce14] + AbstractTrees v0.4.5 [20533458] + BenchmarkCI v0.1.9 [6e4b80f9] + BenchmarkTools v1.6.0 [d1d4a3ce] + BitFlags v0.1.9 [944b1d66] + CodecZlib v0.7.8 [34da2185] + Compat v4.17.0 [f0e56b4a] + ConcurrentUtilities v2.5.0 [187b0558] + ConstructionBase v1.6.0 [adafc99b] + CpuId v0.3.1 [460bff9d] + ExceptionUnwrapping v0.1.11 [bc5e4493] + GitHub v5.10.0 [cd3eb016] + HTTP v1.10.17 [692b3bcd] + JLLWrappers v1.7.0 [682c06a0] + JSON v0.21.4 [1d6d02ad] + LeftChildRightSiblingTrees v0.2.0 [e6f89c97] + LoggingExtras v1.1.0 [1914dd2f] + MacroTools v0.5.16 [739be429] + MbedTLS v1.1.9 [4d8831e6] + OpenSSL v1.5.0 [69de0a69] + Parsers v2.8.3 [32113eaa] + PkgBenchmark v0.2.13 [aea7be01] + PrecompileTools v1.3.2 [21216c6a] + Preferences v1.4.3 [33c8b6b6] + ProgressLogging v0.1.5 [ae029012] + Requires v1.3.1 ⌅ [efcf1570] + Setfield v0.8.3 [777ac1f9] + SimpleBufferStream v1.2.0 [2133526b] + SodiumSeal v0.1.1 [10745b16] + Statistics v1.11.1 [5d786b92] + TerminalLoggers v0.1.7 [3bb67fe8] + TranscodingStreams v0.11.3 [5c2747f8] + URIs v1.6.1 [3a884ed6] + UnPack v1.0.2 [c8ffd9c3] + MbedTLS_jll v2.28.6+2 [3161d3a3] + Zstd_jll v1.5.7+1 [a9144af2] + libsodium_jll v1.0.21+0 [0dad84c5] + ArgTools v1.1.2 [56f22d72] + Artifacts v1.11.0 [2a0f44e3] + Base64 v1.11.0 [ade2ca70] + Dates v1.11.0 [f43a241f] + Downloads v1.6.0 [7b1f6079] + FileWatching v1.11.0 [9fa8497b] + Future v1.11.0 [b77e0a4c] + InteractiveUtils v1.11.0 [ac6e5ff7] + JuliaSyntaxHighlighting v1.12.0 [b27032c2] + LibCURL v0.6.4 [76f85450] + LibGit2 v1.11.0 [8f399da3] + Libdl v1.11.0 [37e2e46d] + LinearAlgebra v1.12.0 [56ddb016] + Logging v1.11.0 [d6f4376e] + Markdown v1.11.0 [a63ad114] + Mmap v1.11.0 [ca575930] + NetworkOptions v1.3.0 [44cfe95a] + Pkg v1.13.0 [de0858da] + Printf v1.11.0 [9abbd945] + Profile v1.11.0 [9a3f8284] + Random v1.11.0 [ea8e919c] + SHA v0.7.0 [9e88b42a] + Serialization v1.11.0 [6462fe0b] + Sockets v1.11.0 [f489334b] + StyledStrings v1.11.0 [fa267f1f] + TOML v1.0.3 [a4e569a6] + Tar v1.10.0 [8dfed614] + Test v1.11.0 [cf7118a7] + UUIDs v1.11.0 [4ec0a83e] + Unicode v1.11.0 [e66e0078] + CompilerSupportLibraries_jll v1.3.0+1 [deac9b47] + LibCURL_jll v8.11.1+1 [e37daf67] + LibGit2_jll v1.9.0+0 [29816b5a] + LibSSH2_jll v1.11.3+1 [14a3606d] + MozillaCACerts_jll v2025.5.20 [4536629a] + OpenBLAS_jll v0.3.29+0 [458c3c95] + OpenSSL_jll v3.5.1+0 [83775a58] + Zlib_jll v1.3.1+2 [8e850b90] + libblastrampoline_jll v5.13.1+0 [8e850ede] + nghttp2_jll v1.64.0+1 [3f19e933] + p7zip_jll v17.5.0+2 Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m` Installation completed after 4.24s ################################################################################ # Precompilation # Precompiling PkgEval dependencies... Precompiling package dependencies... Precompilation completed after 35.17s ################################################################################ # Testing # Testing BenchmarkCI Status `/tmp/jl_Cc4dGG/Project.toml` [20533458] BenchmarkCI v0.1.9 [6e4b80f9] BenchmarkTools v1.6.0 [adafc99b] CpuId v0.3.1 [bc5e4493] GitHub v5.10.0 [682c06a0] JSON v0.21.4 [32113eaa] PkgBenchmark v0.2.13 ⌅ [efcf1570] Setfield v0.8.3 [3a884ed6] UnPack v1.0.2 [3161d3a3] Zstd_jll v1.5.7+1 [2a0f44e3] Base64 v1.11.0 [ade2ca70] Dates v1.11.0 [37e2e46d] LinearAlgebra v1.12.0 [56ddb016] Logging v1.11.0 [d6f4376e] Markdown v1.11.0 [44cfe95a] Pkg v1.13.0 [a4e569a6] Tar v1.10.0 [8dfed614] Test v1.11.0 Status `/tmp/jl_Cc4dGG/Manifest.toml` [1520ce14] AbstractTrees v0.4.5 [20533458] BenchmarkCI v0.1.9 [6e4b80f9] BenchmarkTools v1.6.0 [d1d4a3ce] BitFlags v0.1.9 [944b1d66] CodecZlib v0.7.8 [34da2185] Compat v4.17.0 [f0e56b4a] ConcurrentUtilities v2.5.0 [187b0558] ConstructionBase v1.6.0 [adafc99b] CpuId v0.3.1 [460bff9d] ExceptionUnwrapping v0.1.11 [bc5e4493] GitHub v5.10.0 [cd3eb016] HTTP v1.10.17 [692b3bcd] JLLWrappers v1.7.0 [682c06a0] JSON v0.21.4 [1d6d02ad] LeftChildRightSiblingTrees v0.2.0 [e6f89c97] LoggingExtras v1.1.0 [1914dd2f] MacroTools v0.5.16 [739be429] MbedTLS v1.1.9 [4d8831e6] OpenSSL v1.5.0 [69de0a69] Parsers v2.8.3 [32113eaa] PkgBenchmark v0.2.13 [aea7be01] PrecompileTools v1.3.2 [21216c6a] Preferences v1.4.3 [33c8b6b6] ProgressLogging v0.1.5 [ae029012] Requires v1.3.1 ⌅ [efcf1570] Setfield v0.8.3 [777ac1f9] SimpleBufferStream v1.2.0 [2133526b] SodiumSeal v0.1.1 [10745b16] Statistics v1.11.1 [5d786b92] TerminalLoggers v0.1.7 [3bb67fe8] TranscodingStreams v0.11.3 [5c2747f8] URIs v1.6.1 [3a884ed6] UnPack v1.0.2 [c8ffd9c3] MbedTLS_jll v2.28.6+2 [3161d3a3] Zstd_jll v1.5.7+1 [a9144af2] libsodium_jll v1.0.21+0 [0dad84c5] ArgTools v1.1.2 [56f22d72] Artifacts v1.11.0 [2a0f44e3] Base64 v1.11.0 [ade2ca70] Dates v1.11.0 [f43a241f] Downloads v1.6.0 [7b1f6079] FileWatching v1.11.0 [9fa8497b] Future v1.11.0 [b77e0a4c] InteractiveUtils v1.11.0 [ac6e5ff7] JuliaSyntaxHighlighting v1.12.0 [b27032c2] LibCURL v0.6.4 [76f85450] LibGit2 v1.11.0 [8f399da3] Libdl v1.11.0 [37e2e46d] LinearAlgebra v1.12.0 [56ddb016] Logging v1.11.0 [d6f4376e] Markdown v1.11.0 [a63ad114] Mmap v1.11.0 [ca575930] NetworkOptions v1.3.0 [44cfe95a] Pkg v1.13.0 [de0858da] Printf v1.11.0 [9abbd945] Profile v1.11.0 [9a3f8284] Random v1.11.0 [ea8e919c] SHA v0.7.0 [9e88b42a] Serialization v1.11.0 [6462fe0b] Sockets v1.11.0 [f489334b] StyledStrings v1.11.0 [fa267f1f] TOML v1.0.3 [a4e569a6] Tar v1.10.0 [8dfed614] Test v1.11.0 [cf7118a7] UUIDs v1.11.0 [4ec0a83e] Unicode v1.11.0 [e66e0078] CompilerSupportLibraries_jll v1.3.0+1 [deac9b47] LibCURL_jll v8.11.1+1 [e37daf67] LibGit2_jll v1.9.0+0 [29816b5a] LibSSH2_jll v1.11.3+1 [14a3606d] MozillaCACerts_jll v2025.5.20 [4536629a] OpenBLAS_jll v0.3.29+0 [458c3c95] OpenSSL_jll v3.5.1+0 [83775a58] Zlib_jll v1.3.1+2 [8e850b90] libblastrampoline_jll v5.13.1+0 [8e850ede] nghttp2_jll v1.64.0+1 [3f19e933] p7zip_jll v17.5.0+2 Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. Testing Running tests... Cloning into 'BenchmarkCIExample.jl'... [ Info: Testing with target = testcase/0000-with-manifest Note: switching to 'testcase/0000-with-manifest'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false HEAD is now at 4f8cbb1 Add benchmark/{Project,Manifest}.toml ERROR: MethodError: no method matching print(::Dict{Symbol, Any}) You may have intended to import Base.print The applicable method may be too new: running in world age 38507, while current world is 38517. Closest candidates are: print(::Any, !Matched::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:385 print(::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:386 print(!Matched::IO, !Matched::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:383 ... Stacktrace: [1] top-level scope @ none:11 [2] eval(m::Module, e::Any) @ Core ./boot.jl:489 [3] exec_options(opts::Base.JLOptions) @ Base ./client.jl:287 [4] _start() @ Base ./client.jl:554 default: Error During Test at /home/pkgeval/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:61 Got exception outside of a @test Unexpected end of input ...when parsing byte with value '0' Stacktrace: [1] error(s::String) @ Base ./error.jl:44 [2] _error(message::String, ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:148 [3] byteat(ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:57 [4] current @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:70 [inlined] [5] chomp_space! @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:115 [inlined] [6] parse_value(pc::JSON.Parser.ParserContext{Dict{String, Any}, Int64, true, nothing}, ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:158 [7] parse(io::Base.Process; dicttype::Type, inttype::Type{Int64}, allownan::Bool, null::Nothing) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:481 [8] parse @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:474 [inlined] [9] open(::typeof(JSON.Parser.parse), ::Cmd; kwargs::@Kwargs{}) @ Base ./process.jl:447 [10] open @ ./process.jl:428 [inlined] [11] julia_info @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:150 [inlined] [12] metadata_from(; target::PkgBenchmark.BenchmarkConfig, baseline::PkgBenchmark.BenchmarkConfig, pkgdir::String, script::String, project::String) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:154 [13] judge(target::Nothing, baseline::String; workspace::String, pkgdir::String, script::String, project::String, logger_factory::Type, kwargs::@Kwargs{}) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:312 [14] judge(target::Nothing, baseline::String) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:290 [15] judge @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:287 [inlined] [16] runall @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:781 [inlined] [17] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:62 [inlined] [18] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [19] (::Main.TestBenchmarkCI.TestIntegration.var"#9#10")() @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:62 [20] withenv(::Main.TestBenchmarkCI.TestIntegration.var"#9#10", ::Pair{String, String}, ::Vararg{Pair{String}}) @ Base ./env.jl:265 [21] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:60 [inlined] [22] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [23] (::Main.TestBenchmarkCI.TestIntegration.var"#runtests#8"{Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"})(target::String) @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:58 [24] (::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"})() @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:73 [25] cd(f::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, dir::String) @ Base.Filesystem ./file.jl:112 [26] #4 @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:47 [inlined] [27] mktempdir(fn::Main.TestBenchmarkCI.TestIntegration.var"#4#5"{Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, parent::String; prefix::String) @ Base.Filesystem ./file.jl:899 [28] top-level scope @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:33 [29] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [30] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:46 [inlined] [31] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:306 [32] top-level scope @ ~/.julia/packages/BenchmarkCI/jAAT2/test/runtests.jl:4 [33] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1865 [inlined] [34] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/runtests.jl:7 [inlined] [35] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:306 [36] top-level scope @ none:6 [37] eval(m::Module, e::Any) @ Core ./boot.jl:489 [38] exec_options(opts::Base.JLOptions) @ Base ./client.jl:287 [39] _start() @ Base ./client.jl:554 ERROR: MethodError: no method matching print(::Dict{Symbol, Any}) You may have intended to import Base.print The applicable method may be too new: running in world age 38507, while current world is 38517. Closest candidates are: print(::Any, !Matched::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:385 print(::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:386 print(!Matched::IO, !Matched::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:383 ... Stacktrace: [1] top-level scope @ none:11 [2] eval(m::Module, e::Any) @ Core ./boot.jl:489 [3] exec_options(opts::Base.JLOptions) @ Base ./client.jl:287 [4] _start() @ Base ./client.jl:554 project = benchmark/Project.toml: Error During Test at /home/pkgeval/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:65 Got exception outside of a @test Unexpected end of input ...when parsing byte with value '0' Stacktrace: [1] error(s::String) @ Base ./error.jl:44 [2] _error(message::String, ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:148 [3] byteat(ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:57 [4] current @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:70 [inlined] [5] chomp_space! @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:115 [inlined] [6] parse_value(pc::JSON.Parser.ParserContext{Dict{String, Any}, Int64, true, nothing}, ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:158 [7] parse(io::Base.Process; dicttype::Type, inttype::Type{Int64}, allownan::Bool, null::Nothing) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:481 [8] parse @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:474 [inlined] [9] open(::typeof(JSON.Parser.parse), ::Cmd; kwargs::@Kwargs{}) @ Base ./process.jl:447 [10] open @ ./process.jl:428 [inlined] [11] julia_info @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:150 [inlined] [12] metadata_from(; target::PkgBenchmark.BenchmarkConfig, baseline::PkgBenchmark.BenchmarkConfig, pkgdir::String, script::String, project::String) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:154 [13] judge(target::Nothing, baseline::String; workspace::String, pkgdir::String, script::String, project::String, logger_factory::Type, kwargs::@Kwargs{}) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:312 [14] judge @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:290 [inlined] [15] judge @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:287 [inlined] [16] runall(; kwargs::@Kwargs{project::String}) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:781 [17] runall @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:781 [inlined] [18] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:66 [inlined] [19] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [20] (::Main.TestBenchmarkCI.TestIntegration.var"#9#10")() @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:66 [21] withenv(::Main.TestBenchmarkCI.TestIntegration.var"#9#10", ::Pair{String, String}, ::Vararg{Pair{String}}) @ Base ./env.jl:265 [22] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:60 [inlined] [23] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [24] (::Main.TestBenchmarkCI.TestIntegration.var"#runtests#8"{Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"})(target::String) @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:58 [25] (::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"})() @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:73 [26] cd(f::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, dir::String) @ Base.Filesystem ./file.jl:112 [27] #4 @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:47 [inlined] [28] mktempdir(fn::Main.TestBenchmarkCI.TestIntegration.var"#4#5"{Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, parent::String; prefix::String) @ Base.Filesystem ./file.jl:899 [29] top-level scope @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:33 [30] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [31] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:46 [inlined] [32] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:306 [33] top-level scope @ ~/.julia/packages/BenchmarkCI/jAAT2/test/runtests.jl:4 [34] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1865 [inlined] [35] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/runtests.jl:7 [inlined] [36] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:306 [37] top-level scope @ none:6 [38] eval(m::Module, e::Any) @ Core ./boot.jl:489 [39] exec_options(opts::Base.JLOptions) @ Base ./client.jl:287 [40] _start() @ Base ./client.jl:554 [ Info: Testing with target = testcase/0001-without-manifest Previous HEAD position was 4f8cbb1 Add benchmark/{Project,Manifest}.toml HEAD is now at 5e619c2 Use BenchmarkCI without benchmark/Manifest.toml Removing .benchmarkci/ ERROR: MethodError: no method matching print(::Dict{Symbol, Any}) You may have intended to import Base.print The applicable method may be too new: running in world age 38507, while current world is 38517. Closest candidates are: print(::Any, !Matched::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:385 print(::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:386 print(!Matched::IO, !Matched::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:383 ... Stacktrace: [1] top-level scope @ none:11 [2] eval(m::Module, e::Any) @ Core ./boot.jl:489 [3] exec_options(opts::Base.JLOptions) @ Base ./client.jl:287 [4] _start() @ Base ./client.jl:554 default: Error During Test at /home/pkgeval/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:61 Got exception outside of a @test Unexpected end of input ...when parsing byte with value '0' Stacktrace: [1] error(s::String) @ Base ./error.jl:44 [2] _error(message::String, ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:148 [3] byteat(ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:57 [4] current @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:70 [inlined] [5] chomp_space! @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:115 [inlined] [6] parse_value(pc::JSON.Parser.ParserContext{Dict{String, Any}, Int64, true, nothing}, ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:158 [7] parse(io::Base.Process; dicttype::Type, inttype::Type{Int64}, allownan::Bool, null::Nothing) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:481 [8] parse @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:474 [inlined] [9] open(::typeof(JSON.Parser.parse), ::Cmd; kwargs::@Kwargs{}) @ Base ./process.jl:447 [10] open @ ./process.jl:428 [inlined] [11] julia_info @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:150 [inlined] [12] metadata_from(; target::PkgBenchmark.BenchmarkConfig, baseline::PkgBenchmark.BenchmarkConfig, pkgdir::String, script::String, project::String) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:154 [13] judge(target::Nothing, baseline::String; workspace::String, pkgdir::String, script::String, project::String, logger_factory::Type, kwargs::@Kwargs{}) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:312 [14] judge(target::Nothing, baseline::String) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:290 [15] judge @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:287 [inlined] [16] runall @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:781 [inlined] [17] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:62 [inlined] [18] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [19] (::Main.TestBenchmarkCI.TestIntegration.var"#9#10")() @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:62 [20] withenv(::Main.TestBenchmarkCI.TestIntegration.var"#9#10", ::Pair{String, String}, ::Vararg{Pair{String}}) @ Base ./env.jl:265 [21] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:60 [inlined] [22] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [23] (::Main.TestBenchmarkCI.TestIntegration.var"#runtests#8"{Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"})(target::String) @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:58 [24] (::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"})() @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:74 [25] cd(f::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, dir::String) @ Base.Filesystem ./file.jl:112 [26] #4 @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:47 [inlined] [27] mktempdir(fn::Main.TestBenchmarkCI.TestIntegration.var"#4#5"{Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, parent::String; prefix::String) @ Base.Filesystem ./file.jl:899 [28] top-level scope @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:33 [29] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [30] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:46 [inlined] [31] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:306 [32] top-level scope @ ~/.julia/packages/BenchmarkCI/jAAT2/test/runtests.jl:4 [33] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1865 [inlined] [34] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/runtests.jl:7 [inlined] [35] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:306 [36] top-level scope @ none:6 [37] eval(m::Module, e::Any) @ Core ./boot.jl:489 [38] exec_options(opts::Base.JLOptions) @ Base ./client.jl:287 [39] _start() @ Base ./client.jl:554 ERROR: MethodError: no method matching print(::Dict{Symbol, Any}) You may have intended to import Base.print The applicable method may be too new: running in world age 38507, while current world is 38517. Closest candidates are: print(::Any, !Matched::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:385 print(::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:386 print(!Matched::IO, !Matched::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:383 ... Stacktrace: [1] top-level scope @ none:11 [2] eval(m::Module, e::Any) @ Core ./boot.jl:489 [3] exec_options(opts::Base.JLOptions) @ Base ./client.jl:287 [4] _start() @ Base ./client.jl:554 project = benchmark/Project.toml: Error During Test at /home/pkgeval/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:65 Got exception outside of a @test Unexpected end of input ...when parsing byte with value '0' Stacktrace: [1] error(s::String) @ Base ./error.jl:44 [2] _error(message::String, ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:148 [3] byteat(ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:57 [4] current @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:70 [inlined] [5] chomp_space! @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:115 [inlined] [6] parse_value(pc::JSON.Parser.ParserContext{Dict{String, Any}, Int64, true, nothing}, ps::JSON.Parser.StreamingParserState{Base.Process}) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:158 [7] parse(io::Base.Process; dicttype::Type, inttype::Type{Int64}, allownan::Bool, null::Nothing) @ JSON.Parser ~/.julia/packages/JSON/93Ea8/src/Parser.jl:481 [8] parse @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:474 [inlined] [9] open(::typeof(JSON.Parser.parse), ::Cmd; kwargs::@Kwargs{}) @ Base ./process.jl:447 [10] open @ ./process.jl:428 [inlined] [11] julia_info @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:150 [inlined] [12] metadata_from(; target::PkgBenchmark.BenchmarkConfig, baseline::PkgBenchmark.BenchmarkConfig, pkgdir::String, script::String, project::String) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:154 [13] judge(target::Nothing, baseline::String; workspace::String, pkgdir::String, script::String, project::String, logger_factory::Type, kwargs::@Kwargs{}) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:312 [14] judge @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:290 [inlined] [15] judge @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:287 [inlined] [16] runall(; kwargs::@Kwargs{project::String}) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:781 [17] runall @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:781 [inlined] [18] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:66 [inlined] [19] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [20] (::Main.TestBenchmarkCI.TestIntegration.var"#9#10")() @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:66 [21] withenv(::Main.TestBenchmarkCI.TestIntegration.var"#9#10", ::Pair{String, String}, ::Vararg{Pair{String}}) @ Base ./env.jl:265 [22] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:60 [inlined] [23] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [24] (::Main.TestBenchmarkCI.TestIntegration.var"#runtests#8"{Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"})(target::String) @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:58 [25] (::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"})() @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:74 [26] cd(f::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, dir::String) @ Base.Filesystem ./file.jl:112 [27] #4 @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:47 [inlined] [28] mktempdir(fn::Main.TestBenchmarkCI.TestIntegration.var"#4#5"{Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, parent::String; prefix::String) @ Base.Filesystem ./file.jl:899 [29] top-level scope @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:33 [30] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [31] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:46 [inlined] [32] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:306 [33] top-level scope @ ~/.julia/packages/BenchmarkCI/jAAT2/test/runtests.jl:4 [34] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1865 [inlined] [35] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/runtests.jl:7 [inlined] [36] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:306 [37] top-level scope @ none:6 [38] eval(m::Module, e::Any) @ Core ./boot.jl:489 [39] exec_options(opts::Base.JLOptions) @ Base ./client.jl:287 [40] _start() @ Base ./client.jl:554 ERROR: MethodError: no method matching print(::Dict{Symbol, Any}) You may have intended to import Base.print The applicable method may be too new: running in world age 38507, while current world is 38517. Closest candidates are: print(::Any, !Matched::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:385 print(::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:386 print(!Matched::IO, !Matched::Any) (method too new to be called from this world context.) @ JSON ~/.julia/packages/JSON/93Ea8/src/Writer.jl:383 ... Stacktrace: [1] top-level scope @ none:11 [2] eval(m::Module, e::Any) @ Core ./boot.jl:489 [3] exec_options(opts::Base.JLOptions) @ Base ./client.jl:287 [4] _start() @ Base ./client.jl:554 BenchmarkCI.jl: Test Failed at /home/pkgeval/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:84 Expression: occursin("One of the following files must exist:", sprint(showerror, err)) Evaluated: occursin("One of the following files must exist:", "Unexpected end of input\n ...when parsing byte with value '0'") Stacktrace: [1] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:680 [inlined] [2] (::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"})() @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:84 [3] cd(f::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, dir::String) @ Base.Filesystem ./file.jl:112 [4] #4 @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:47 [inlined] [5] mktempdir(fn::Main.TestBenchmarkCI.TestIntegration.var"#4#5"{Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, parent::String; prefix::String) @ Base.Filesystem ./file.jl:899 [6] top-level scope @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:33 [7] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [8] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:46 [inlined] BenchmarkCI.jl: Error During Test at /home/pkgeval/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:32 Got exception outside of a @test SystemError: opening file ".benchmarkci/result-target.json": No such file or directory Stacktrace: [1] systemerror(p::String, errno::Int32; extrainfo::Nothing) @ Base ./error.jl:186 [2] open(fname::String; lock::Bool, read::Nothing, write::Nothing, create::Nothing, truncate::Nothing, append::Nothing) @ Base ./iostream.jl:317 [3] open @ ./iostream.jl:296 [inlined] [4] open(f::JSON.Parser.var"#5#6"{Type{Dict{String, Any}}, Type{Int64}, Nothing, Bool, Bool, Int64}, args::String; kwargs::@Kwargs{}) @ Base ./io.jl:408 [5] open @ ./io.jl:407 [inlined] [6] #parsefile#3 @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:509 [inlined] [7] parsefile @ ~/.julia/packages/JSON/93Ea8/src/Parser.jl:502 [inlined] [8] readresults(file::String) @ PkgBenchmark ~/.julia/packages/PkgBenchmark/15w1x/src/benchmarkresults.jl:78 [9] _loadjudge(workspace::String) @ BenchmarkCI ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:451 [10] _loadciresult (repeats 2 times) @ ~/.julia/packages/BenchmarkCI/jAAT2/src/BenchmarkCI.jl:457 [inlined] [11] (::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"})() @ Main.TestBenchmarkCI.TestIntegration ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:86 [12] cd(f::Main.TestBenchmarkCI.TestIntegration.var"#6#7"{String, Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, dir::String) @ Base.Filesystem ./file.jl:112 [13] #4 @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:47 [inlined] [14] mktempdir(fn::Main.TestBenchmarkCI.TestIntegration.var"#4#5"{Main.TestBenchmarkCI.TestIntegration.var"#printlns#printlns##0"{Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, Main.TestBenchmarkCI.TestIntegration.var"#flushall#flushall##0"}, parent::String; prefix::String) @ Base.Filesystem ./file.jl:899 [15] top-level scope @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:33 [16] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1776 [inlined] [17] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/test_integration.jl:46 [inlined] [18] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:306 [19] top-level scope @ ~/.julia/packages/BenchmarkCI/jAAT2/test/runtests.jl:4 [20] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1865 [inlined] [21] macro expansion @ ~/.julia/packages/BenchmarkCI/jAAT2/test/runtests.jl:7 [inlined] [22] include(mapexpr::Function, mod::Module, _path::String) @ Base ./Base.jl:306 [23] top-level scope @ none:6 [24] eval(m::Module, e::Any) @ Core ./boot.jl:489 [25] exec_options(opts::Base.JLOptions) @ Base ./client.jl:287 [26] _start() @ Base ./client.jl:554 Test Summary: | Pass Fail Error Total Time test_integration.jl | 1 1 5 7 1m39.4s BenchmarkCI.jl | 1 1 5 7 1m32.0s testcase/0000-with-manifest | 2 2 29.4s default | 1 1 17.1s project = benchmark/Project.toml | 1 1 12.3s testcase/0001-without-manifest | 2 2 24.2s default | 1 1 12.3s project = benchmark/Project.toml | 1 1 11.9s RNG of the outermost testset: Random.Xoshiro(0xe877f56b01f72a98, 0xb994dfbe887305e2, 0x09565fe16d9d7132, 0x62f78d475e2620ef, 0xf8ab5101526e58c1) ERROR: LoadError: Some tests did not pass: 1 passed, 1 failed, 5 errored, 0 broken. in expression starting at /home/pkgeval/.julia/packages/BenchmarkCI/jAAT2/test/runtests.jl:1 Testing failed after 110.28s ERROR: LoadError: Package BenchmarkCI errored during testing Stacktrace: [1] pkgerror(msg::String) @ Pkg.Types /opt/julia/share/julia/stdlib/v1.12/Pkg/src/Types.jl:68 [2] test(ctx::Pkg.Types.Context, pkgs::Vector{PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool) @ Pkg.Operations /opt/julia/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:2458 [3] test @ /opt/julia/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:2313 [inlined] [4] test(ctx::Pkg.Types.Context, pkgs::Vector{PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::@Kwargs{io::IOContext{IO}}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:511 [5] test(pkgs::Vector{PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{julia_args::Cmd}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:164 [6] test(pkgs::Vector{String}; kwargs::@Kwargs{julia_args::Cmd}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:152 [7] test @ /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:152 [inlined] [8] #test#81 @ /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:151 [inlined] [9] top-level scope @ /PkgEval.jl/scripts/evaluate.jl:219 [10] include(mod::Module, _path::String) @ Base ./Base.jl:305 [11] exec_options(opts::Base.JLOptions) @ Base ./client.jl:321 [12] _start() @ Base ./client.jl:554 in expression starting at /PkgEval.jl/scripts/evaluate.jl:210 PkgEval failed after 182.24s: package tests unexpectedly errored