Package evaluation of PkgJogger on Julia 1.12.0-DEV.2126 (3761029c84*) started at 2025-03-25T13:01:09.414 ################################################################################ # Set-up # Installing PkgEval dependencies (TestEnv)... Set-up completed after 9.05s ################################################################################ # Installation # Installing PkgJogger... Resolving package versions... Installed Parsers ──────────── v2.8.1 Installed JSON ─────────────── v0.21.4 Installed PkgJogger ────────── v0.5.1 Installed Statistics ───────── v1.11.1 Installed Preferences ──────── v1.4.3 Installed TranscodingStreams ─ v0.11.3 Installed PrecompileTools ──── v1.2.1 Installed BenchmarkTools ───── v1.6.0 Installed BSON ─────────────── v0.3.9 Installed CodecZlib ────────── v0.7.8 Installed Compat ───────────── v4.16.0 Updating `~/.julia/environments/v1.12/Project.toml` [10150987] + PkgJogger v0.5.1 Updating `~/.julia/environments/v1.12/Manifest.toml` [fbb218c0] + BSON v0.3.9 [6e4b80f9] + BenchmarkTools v1.6.0 [944b1d66] + CodecZlib v0.7.8 [34da2185] + Compat v4.16.0 [682c06a0] + JSON v0.21.4 [69de0a69] + Parsers v2.8.1 [10150987] + PkgJogger v0.5.1 [aea7be01] + PrecompileTools v1.2.1 [21216c6a] + Preferences v1.4.3 [10745b16] + Statistics v1.11.1 [3bb67fe8] + TranscodingStreams v0.11.3 [0dad84c5] + ArgTools v1.1.2 [56f22d72] + Artifacts v1.11.0 [2a0f44e3] + Base64 v1.11.0 [ade2ca70] + Dates v1.11.0 [f43a241f] + Downloads v1.6.0 [7b1f6079] + FileWatching v1.11.0 [b77e0a4c] + InteractiveUtils v1.11.0 [ac6e5ff7] + JuliaSyntaxHighlighting v1.12.0 [b27032c2] + LibCURL v0.6.4 [76f85450] + LibGit2 v1.11.0 [8f399da3] + Libdl v1.11.0 [37e2e46d] + LinearAlgebra v1.12.0 [56ddb016] + Logging v1.11.0 [d6f4376e] + Markdown v1.11.0 [a63ad114] + Mmap v1.11.0 [ca575930] + NetworkOptions v1.3.0 [44cfe95a] + Pkg v1.12.0 [de0858da] + Printf v1.11.0 [9abbd945] + Profile v1.11.0 [9a3f8284] + Random v1.11.0 [ea8e919c] + SHA v0.7.0 [9e88b42a] + Serialization v1.11.0 [f489334b] + StyledStrings v1.11.0 [fa267f1f] + TOML v1.0.3 [a4e569a6] + Tar v1.10.0 [8dfed614] + Test v1.11.0 [cf7118a7] + UUIDs v1.11.0 [4ec0a83e] + Unicode v1.11.0 [e66e0078] + CompilerSupportLibraries_jll v1.3.0+1 [deac9b47] + LibCURL_jll v8.11.1+1 [e37daf67] + LibGit2_jll v1.9.0+0 [29816b5a] + LibSSH2_jll v1.11.3+1 [14a3606d] + MozillaCACerts_jll v2024.12.31 [4536629a] + OpenBLAS_jll v0.3.29+0 [458c3c95] + OpenSSL_jll v3.0.16+0 [83775a58] + Zlib_jll v1.3.1+2 [8e850b90] + libblastrampoline_jll v5.12.0+0 [8e850ede] + nghttp2_jll v1.64.0+1 [3f19e933] + p7zip_jll v17.5.0+2 Installation completed after 1.44s ################################################################################ # Precompilation # Precompiling PkgEval dependencies... Precompiling packages... 5118.4 ms ✓ TestEnv 1 dependency successfully precompiled in 7 seconds. 25 already precompiled. Precompiling package dependencies... Precompilation completed after 148.97s ################################################################################ # Testing # Testing PkgJogger Status `/tmp/jl_tJ5jbG/Project.toml` [fbb218c0] BSON v0.3.9 [6e4b80f9] BenchmarkTools v1.6.0 [944b1d66] CodecZlib v0.7.8 [682c06a0] JSON v0.21.4 [10150987] PkgJogger v0.5.1 [817f1d60] ReTestItems v1.29.0 [295af30f] Revise v3.7.3 [10745b16] Statistics v1.11.1 [ade2ca70] Dates v1.11.0 [76f85450] LibGit2 v1.11.0 [44cfe95a] Pkg v1.12.0 [9a3f8284] Random v1.11.0 [fa267f1f] TOML v1.0.3 [8dfed614] Test v1.11.0 [cf7118a7] UUIDs v1.11.0 Status `/tmp/jl_tJ5jbG/Manifest.toml` [fbb218c0] BSON v0.3.9 [6e4b80f9] BenchmarkTools v1.6.0 [da1fd8a2] CodeTracking v1.3.6 [944b1d66] CodecZlib v0.7.8 [34da2185] Compat v4.16.0 [682c06a0] JSON v0.21.4 [aa1ae85d] JuliaInterpreter v0.9.42 [e6f89c97] LoggingExtras v1.1.0 [6f1432cf] LoweredCodeUtils v3.1.0 [bac558e1] OrderedCollections v1.8.0 [69de0a69] Parsers v2.8.1 [10150987] PkgJogger v0.5.1 [aea7be01] PrecompileTools v1.2.1 [21216c6a] Preferences v1.4.3 [817f1d60] ReTestItems v1.29.0 [ae029012] Requires v1.3.1 [295af30f] Revise v3.7.3 [10745b16] Statistics v1.11.1 [1e6cf692] TestEnv v1.102.0 [3bb67fe8] TranscodingStreams v0.11.3 [0dad84c5] ArgTools v1.1.2 [56f22d72] Artifacts v1.11.0 [2a0f44e3] Base64 v1.11.0 [ade2ca70] Dates v1.11.0 [f43a241f] Downloads v1.6.0 [7b1f6079] FileWatching v1.11.0 [b77e0a4c] InteractiveUtils v1.11.0 [ac6e5ff7] JuliaSyntaxHighlighting v1.12.0 [b27032c2] LibCURL v0.6.4 [76f85450] LibGit2 v1.11.0 [8f399da3] Libdl v1.11.0 [37e2e46d] LinearAlgebra v1.12.0 [56ddb016] Logging v1.11.0 [d6f4376e] Markdown v1.11.0 [a63ad114] Mmap v1.11.0 [ca575930] NetworkOptions v1.3.0 [44cfe95a] Pkg v1.12.0 [de0858da] Printf v1.11.0 [9abbd945] Profile v1.11.0 [3fa0cd96] REPL v1.11.0 [9a3f8284] Random v1.11.0 [ea8e919c] SHA v0.7.0 [9e88b42a] Serialization v1.11.0 [6462fe0b] Sockets v1.11.0 [f489334b] StyledStrings v1.11.0 [fa267f1f] TOML v1.0.3 [a4e569a6] Tar v1.10.0 [8dfed614] Test v1.11.0 [cf7118a7] UUIDs v1.11.0 [4ec0a83e] Unicode v1.11.0 [e66e0078] CompilerSupportLibraries_jll v1.3.0+1 [deac9b47] LibCURL_jll v8.11.1+1 [e37daf67] LibGit2_jll v1.9.0+0 [29816b5a] LibSSH2_jll v1.11.3+1 [14a3606d] MozillaCACerts_jll v2024.12.31 [4536629a] OpenBLAS_jll v0.3.29+0 [458c3c95] OpenSSL_jll v3.0.16+0 [83775a58] Zlib_jll v1.3.1+2 [8e850b90] libblastrampoline_jll v5.12.0+0 [8e850ede] nghttp2_jll v1.64.0+1 [3f19e933] p7zip_jll v17.5.0+2 Testing Running tests... ┌ Warning: Verbosity logging macros are deprecated as they are not compatible with juliac-compiled programs │ caller = ip:0x0 └ @ Core :-1 [ Info: Scanning for test items in project `PkgJogger` at paths: /home/pkgeval/.julia/packages/PkgJogger/3NQMW [ Info: Finished scanning for test items in 17.69 seconds. Scheduling 9 tests on pid 119 13:04:54 | maxrss 2.2% | mem 2.4% | START (1/9) test item "compat" at test/backward_compat_test.jl:1 13:05:58 | maxrss 2.2% | mem 2.4% | DONE (1/9) test item "compat" 58.6 secs (47.8% compile, 3.6% recompile, 15.6% GC), 13.03 M allocs (761.682 MB) 13:05:58 | maxrss 2.2% | mem 2.4% | START (2/9) test item "ci" at test/ci_test.jl:1 13:07:35 | maxrss 2.2% | mem 2.4% | DONE (2/9) test item "ci" 93.6 secs (0.7% compile, <0.1% recompile), 387.94 K allocs (32.050 MB) 13:07:35 | maxrss 2.2% | mem 2.4% | START (3/9) test item "judging" at test/judge_tests.jl:1 13:08:27 | maxrss 2.4% | mem 2.6% | DONE (3/9) test item "judging" 46.9 secs (52.6% compile, <0.1% recompile, 37.0% GC), 14.85 M allocs (801.357 MB) 13:08:27 | maxrss 2.4% | mem 2.6% | START (4/9) test item "default suite" at test/locate_benchmarks_tests.jl:43 13:08:34 | maxrss 2.4% | mem 2.6% | DONE (4/9) test item "default suite" 2.5 secs (30.1% compile, <0.1% recompile), 369.05 K allocs (29.965 MB) 13:08:34 | maxrss 2.4% | mem 2.6% | START (5/9) test item "Add benchmarks" at test/locate_benchmarks_tests.jl:49 13:08:43 | maxrss 2.4% | mem 2.6% | DONE (5/9) test item "Add benchmarks" 4.0 secs (49.9% compile, <0.1% recompile, 2.0% GC), 649.56 K allocs (47.318 MB) Captured Logs for test item "Add benchmarks" at test/locate_benchmarks_tests.jl:49 on worker 119 WARNING: Detected access to binding `##ExamplePkg#283.JogExamples92BrxoY` in a world prior to its definition world. Julia 1.12 has introduced more strict world age semantics for global bindings. !!! This code may malfunction under Revise. !!! This code will error in future versions of Julia. Hint: Add an appropriate `invokelatest` around the access to this binding. [ Info: Creating Examples92BrxoY for testing in /tmp/jl_aiK8SQ Activating project at `/tmp/jl_aiK8SQ` No packages added to or removed from `/tmp/jl_aiK8SQ/Project.toml` No packages added to or removed from `/tmp/jl_aiK8SQ/Manifest.toml` Precompiling packages... 752.0 ms ✓ Examples92BrxoY 1 dependency successfully precompiled in 1 seconds Activating project at `/tmp/jl_tJ5jbG` Error in testset "Add benchmarks" on worker 119: Test Failed at /home/pkgeval/.julia/packages/PkgJogger/3NQMW/test/locate_benchmarks_tests.jl:21 Expression: suite == reference Evaluated: Set(Vector{Any}[["bench_timer.jl", "1ms"], ["bench_timer.jl", "2ms"]]) == Set([["bench_foo_9601.jl", "foo"], ["bench_timer.jl", "1ms"], ["bench_timer.jl", "2ms"]]) Error in testset "Add benchmarks" on worker 119: Test Failed at /home/pkgeval/.julia/packages/PkgJogger/3NQMW/test/locate_benchmarks_tests.jl:21 Expression: suite == reference Evaluated: Set(Vector{Any}[["bench_timer.jl", "1ms"], ["bench_timer.jl", "2ms"]]) == Set([["bench_foo_9601.jl", "foo"], ["bench_timer.jl", "1ms"], ["bench_timer.jl", "2ms"]]) Error in testset "Add benchmarks" on worker 119: Test Failed at /home/pkgeval/.julia/packages/PkgJogger/3NQMW/test/locate_benchmarks_tests.jl:21 Expression: suite == reference Evaluated: Set(Vector{Any}[["bench_timer.jl", "1ms"], ["bench_timer.jl", "2ms"]]) == Set([["bench_foo_9601.jl", "foo"], ["bench_timer.jl", "1ms"], ["bench_timer.jl", "2ms"]]) Error in testset "Add benchmarks" on worker 119: Test Failed at /home/pkgeval/.julia/packages/PkgJogger/3NQMW/test/locate_benchmarks_tests.jl:21 Expression: suite == reference Evaluated: Set(Vector{Any}[["bench_foo_9601.jl", "foo"], ["bench_timer.jl", "1ms"], ["bench_timer.jl", "2ms"]]) == Set([["bench_foo_9601.jl", "foo"], ["bench_timer.jl", "1ms"], ["bench_timer.jl", "2ms"], ["bench_foo_12821.jl", "foo"]]) 13:08:44 | maxrss 2.4% | mem 2.6% | START (6/9) test item "Benchmarks in subfolders" at test/locate_benchmarks_tests.jl:71 13:08:51 | maxrss 2.4% | mem 2.6% | DONE (6/9) test item "Benchmarks in subfolders" 2.6 secs (5.6% compile, <0.1% recompile), 171.15 K allocs (18.300 MB) Captured Logs for test item "Benchmarks in subfolders" at test/locate_benchmarks_tests.jl:71 on worker 119 WARNING: Detected access to binding `##ExamplePkg#283.JogExampledR28XLwU` in a world prior to its definition world. Julia 1.12 has introduced more strict world age semantics for global bindings. !!! This code may malfunction under Revise. !!! This code will error in future versions of Julia. Hint: Add an appropriate `invokelatest` around the access to this binding. [ Info: Creating ExampledR28XLwU for testing in /tmp/jl_LxNVUk Activating project at `/tmp/jl_LxNVUk` No packages added to or removed from `/tmp/jl_LxNVUk/Project.toml` No packages added to or removed from `/tmp/jl_LxNVUk/Manifest.toml` Precompiling packages... 775.1 ms ✓ ExampledR28XLwU 1 dependency successfully precompiled in 1 seconds Activating project at `/tmp/jl_tJ5jbG` Error in testset "Benchmarks in subfolders" on worker 119: Test Failed at /home/pkgeval/.julia/packages/PkgJogger/3NQMW/test/locate_benchmarks_tests.jl:21 Expression: suite == reference Evaluated: Set(Vector{Any}[["bench_timer.jl", "1ms"], ["bench_timer.jl", "2ms"]]) == Set([["bench_timer.jl", "1ms"], ["bench_subdir_29671", "bench_foo_2716.jl", "foo"], ["bench_timer.jl", "2ms"]]) Error in testset "Benchmarks in subfolders" on worker 119: Test Failed at /home/pkgeval/.julia/packages/PkgJogger/3NQMW/test/locate_benchmarks_tests.jl:21 Expression: suite == reference Evaluated: Set(Vector{Any}[["bench_timer.jl", "1ms"], ["bench_subdir_29671", "bench_foo_2716.jl", "foo"], ["bench_timer.jl", "2ms"]]) == Set([["bench_subdir_46421", "bench_l2", "bench_foo_57300.jl", "foo"], ["bench_timer.jl", "1ms"], ["bench_subdir_29671", "bench_foo_2716.jl", "foo"], ["bench_timer.jl", "2ms"], ["bench_subdir_46421", "bench_foo_44302.jl", "foo"]]) 13:08:51 | maxrss 2.4% | mem 2.6% | START (7/9) test item "canonical" at test/smoke_tests.jl:1 13:09:40 | maxrss 2.4% | mem 2.6% | DONE (7/9) test item "canonical" 43.7 secs (11.3% compile, <0.1% recompile, 79.8% GC), 2.87 M allocs (166.890 MB) 13:09:40 | maxrss 2.4% | mem 2.6% | START (8/9) test item "benchmark and save" at test/smoke_tests.jl:83 13:09:56 | maxrss 2.4% | mem 2.6% | DONE (8/9) test item "benchmark and save" 11.8 secs (2.7% compile, <0.1% recompile, 73.3% GC), 729.23 K allocs (50.761 MB) 13:09:56 | maxrss 2.4% | mem 2.6% | START (9/9) test item "unit tests" at test/tune_tests.jl:32 13:10:32 | maxrss 2.4% | mem 2.6% | DONE (9/9) test item "unit tests" 31.1 secs (11.5% compile, <0.1% recompile, 77.0% GC), 1.18 M allocs (77.199 MB) Test Summary: | Pass Fail Total Time PkgJogger | 128 6 134 6m01.0s test | 128 6 134 test/backward_compat_test.jl | 4 4 test/ci_test.jl | 1 1 test/judge_tests.jl | 37 37 test/locate_benchmarks_tests.jl | 3 6 9 default suite | 1 1 6.3s Add benchmarks | 4 4 7.8s Benchmarks in subfolders | 2 2 4 6.4s test/smoke_tests.jl | 53 53 test/tune_tests.jl | 30 30 ERROR: LoadError: Some tests did not pass: 128 passed, 6 failed, 0 errored, 0 broken. in expression starting at /home/pkgeval/.julia/packages/PkgJogger/3NQMW/test/runtests.jl:2 Testing failed after 349.92s ERROR: LoadError: Package PkgJogger errored during testing Stacktrace: [1] pkgerror(msg::String) @ Pkg.Types /opt/julia/share/julia/stdlib/v1.12/Pkg/src/Types.jl:68 [2] test(ctx::Pkg.Types.Context, pkgs::Vector{PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool) @ Pkg.Operations /opt/julia/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:2365 [3] test @ /opt/julia/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:2220 [inlined] [4] test(ctx::Pkg.Types.Context, pkgs::Vector{PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::@Kwargs{io::IOContext{IO}}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:486 [5] test(pkgs::Vector{PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{julia_args::Cmd}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:164 [6] test(pkgs::Vector{String}; kwargs::@Kwargs{julia_args::Cmd}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:152 [7] test @ /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:152 [inlined] [8] #test#81 @ /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:151 [inlined] [9] top-level scope @ /PkgEval.jl/scripts/evaluate.jl:219 [10] include(mod::Module, _path::String) @ Base ./Base.jl:303 [11] exec_options(opts::Base.JLOptions) @ Base ./client.jl:328 [12] _start() @ Base ./client.jl:560 in expression starting at /PkgEval.jl/scripts/evaluate.jl:210 PkgEval failed after 629.77s: package has test failures