Package evaluation to test Grader on Julia 1.12.7-DEV.42 (6f510b6086*) started at 2026-06-19T12:40:46.543 ################################################################################ # Set-up # Installing PkgEval dependencies (TestEnv)... Activating project at `~/.julia/environments/v1.12` Set-up completed after 8.16s ################################################################################ # Installation # Installing Grader... Resolving package versions... Updating `~/.julia/environments/v1.12/Project.toml` [d440d0cf] + Grader v0.3.1 Updating `~/.julia/environments/v1.12/Manifest.toml` [6912e4f1] + Espresso v0.6.4 [d440d0cf] + Grader v0.3.1 ⌅ [682c06a0] + JSON v0.21.4 ⌅ [bac558e1] + OrderedCollections v1.8.2 ⌅ [d96e819e] + Parameters v0.12.3 [69de0a69] + Parsers v2.8.6 [aea7be01] + PrecompileTools v1.3.4 [21216c6a] + Preferences v1.5.2 [10745b16] + Statistics v1.11.1 [3a884ed6] + UnPack v1.0.2 [56f22d72] + Artifacts v1.11.0 [2a0f44e3] + Base64 v1.11.0 [ade2ca70] + Dates v1.11.0 [b77e0a4c] + InteractiveUtils v1.11.0 [ac6e5ff7] + JuliaSyntaxHighlighting v1.12.0 [8f399da3] + Libdl v1.11.0 [37e2e46d] + LinearAlgebra v1.12.0 [56ddb016] + Logging v1.11.0 [d6f4376e] + Markdown v1.11.0 [a63ad114] + Mmap v1.11.0 [de0858da] + Printf v1.11.0 [9a3f8284] + Random v1.11.0 [ea8e919c] + SHA v0.7.0 [9e88b42a] + Serialization v1.11.0 [f489334b] + StyledStrings v1.11.0 [fa267f1f] + TOML v1.0.3 [8dfed614] + Test v1.11.0 [cf7118a7] + UUIDs v1.11.0 [4ec0a83e] + Unicode v1.11.0 [e66e0078] + CompilerSupportLibraries_jll v1.3.0+1 [4536629a] + OpenBLAS_jll v0.3.29+0 [8e850b90] + libblastrampoline_jll v5.15.0+0 Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m` Installation completed after 5.64s ################################################################################ # Precompilation # Precompiling PkgEval dependencies... Precompiling package dependencies... Precompiling packages... 4037.2 ms ✓ Grader 1 dependency successfully precompiled in 6 seconds. 182 already precompiled. 24 dependencies precompiled but different versions are currently loaded (ArgTools, Base64, Dates, Downloads, JuliaSyntaxHighlighting, LibCURL, LibCURL_jll, LibGit2, LibGit2_jll, LibSSH2_jll, Logging, Markdown, MozillaCACerts_jll, NetworkOptions, OpenSSL_jll, Pkg, Printf, StyledStrings, TOML, Tar, UUIDs, Zlib_jll, nghttp2_jll and p7zip_jll). Restart julia to access the new versions. Otherwise, 116 dependents of these packages may trigger further precompilation to work with the unexpected versions. Precompilation completed after 26.01s ################################################################################ # Testing # Testing Grader Status `/tmp/jl_Fp7cPz/Project.toml` [6912e4f1] Espresso v0.6.4 [d440d0cf] Grader v0.3.1 ⌅ [682c06a0] JSON v0.21.4 ⌅ [d96e819e] Parameters v0.12.3 [91a5bcdd] Plots v1.41.6 [37e2e46d] LinearAlgebra v1.12.0 [d6f4376e] Markdown v1.11.0 [9a3f8284] Random v1.11.0 [8dfed614] Test v1.11.0 Status `/tmp/jl_Fp7cPz/Manifest.toml` [66dad0bd] AliasTables v1.1.3 [d1d4a3ce] BitFlags v0.1.10 [944b1d66] CodecZlib v0.7.8 [35d6a980] ColorSchemes v3.31.0 [3da002f7] ColorTypes v0.12.1 [c3611d14] ColorVectorSpace v0.11.0 [5ae59095] Colors v0.13.1 [f0e56b4a] ConcurrentUtilities v2.5.1 [d38c429a] Contour v0.6.3 [9a962f9c] DataAPI v1.16.0 [864edb3b] DataStructures v0.19.5 [8bb1440f] DelimitedFiles v1.9.1 [ffbed154] DocStringExtensions v0.9.5 [6912e4f1] Espresso v0.6.4 [460bff9d] ExceptionUnwrapping v0.1.11 [c87230d0] FFMPEG v0.4.5 ⌅ [53c48c17] FixedPointNumbers v0.8.6 [1fa38f19] Format v1.3.7 [28b8d3ca] GR v0.73.26 [d440d0cf] Grader v0.3.1 [42e2da0e] Grisu v1.0.2 ⌅ [cd3eb016] HTTP v1.11.0 [92d709cd] IrrationalConstants v0.2.6 [1019f520] JLFzf v0.1.11 [692b3bcd] JLLWrappers v1.8.0 ⌅ [682c06a0] JSON v0.21.4 [b964fa9f] LaTeXStrings v1.4.0 [23fbe1c1] Latexify v0.16.10 [2ab3a3ac] LogExpFunctions v1.0.1 [e6f89c97] LoggingExtras v1.2.0 [1914dd2f] MacroTools v0.5.16 [739be429] MbedTLS v1.1.10 [442fdcdd] Measures v0.3.3 [e1d29d7a] Missings v1.2.0 [77ba4419] NaNMath v1.1.4 [4d8831e6] OpenSSL v1.6.1 ⌅ [bac558e1] OrderedCollections v1.8.2 ⌅ [d96e819e] Parameters v0.12.3 [69de0a69] Parsers v2.8.6 [ccf2f8ad] PlotThemes v3.3.0 [995b91a9] PlotUtils v1.4.4 [91a5bcdd] Plots v1.41.6 [aea7be01] PrecompileTools v1.3.4 [21216c6a] Preferences v1.5.2 [43287f4e] PtrArrays v1.4.0 [3cdcf5f2] RecipesBase v1.3.4 [01d81517] RecipesPipeline v0.6.12 [189a3867] Reexport v1.2.2 [05181044] RelocatableFolders v1.0.1 [ae029012] Requires v1.3.1 [6c6a2e73] Scratch v1.3.0 [992d4aef] Showoff v1.0.3 [777ac1f9] SimpleBufferStream v1.2.0 [a2af1166] SortingAlgorithms v1.2.2 [860ef19b] StableRNGs v1.0.4 [10745b16] Statistics v1.11.1 [82ae8749] StatsAPI v1.8.0 [2913bbd2] StatsBase v0.34.12 [62fd8b95] TensorCore v0.1.1 [3bb67fe8] TranscodingStreams v0.11.3 [5c2747f8] URIs v1.6.1 [3a884ed6] UnPack v1.0.2 [1cfade01] UnicodeFun v0.4.1 [41fe7b60] Unzip v0.2.0 [6e34b625] Bzip2_jll v1.0.9+0 [83423d85] Cairo_jll v1.18.7+0 [ee1fde0b] Dbus_jll v1.16.2+0 [2702e6a9] EpollShim_jll v0.0.20230411+1 [2e619515] Expat_jll v2.8.1+0 [b22a6f82] FFMPEG_jll v8.1.0+0 [a3f928ae] Fontconfig_jll v2.17.1+0 [d7e528f0] FreeType2_jll v2.14.3+1 [559328eb] FriBidi_jll v1.0.17+0 [0656b61e] GLFW_jll v3.4.1+1 [d2c73de3] GR_jll v0.73.26+0 ⌅ [b0724c58] GettextRuntime_jll v0.22.4+0 [61579ee1] Ghostscript_jll v9.55.1+0 [7746bdde] Glib_jll v2.86.3+0 [3b182d85] Graphite2_jll v1.3.16+0 [2e76f6c2] HarfBuzz_jll v8.5.1+0 [aacddb02] JpegTurbo_jll v3.1.5+0 [c1c5ebd0] LAME_jll v3.100.3+0 [88015f11] LERC_jll v4.1.0+0 [1d63c593] LLVMOpenMP_jll v18.1.8+0 ⌅ [e9f186c6] Libffi_jll v3.4.7+0 [7e76a0d4] Libglvnd_jll v1.7.1+1 [94ce4f54] Libiconv_jll v1.18.0+0 [4b2f31a3] Libmount_jll v2.42.0+0 [89763e89] Libtiff_jll v4.7.2+0 [38a345b3] Libuuid_jll v2.42.0+0 [c8ffd9c3] MbedTLS_jll v2.28.1010+0 [e7412a2a] Ogg_jll v1.3.6+0 [91d4177d] Opus_jll v1.6.1+0 [36c8627f] Pango_jll v1.57.1+0 [30392449] Pixman_jll v0.46.4+0 [c0090381] Qt6Base_jll v6.10.2+2 [629bc702] Qt6Declarative_jll v6.10.2+2 [ce943373] Qt6ShaderTools_jll v6.10.2+1 [6de9746b] Qt6Svg_jll v6.10.2+0 [e99dba38] Qt6Wayland_jll v6.10.2+1 [a44049a8] Vulkan_Loader_jll v1.3.243+0 [a2964d1f] Wayland_jll v1.24.0+0 [ffd25f8a] XZ_jll v5.8.3+0 [f67eecfb] Xorg_libICE_jll v1.1.2+0 [c834827a] Xorg_libSM_jll v1.2.6+0 [4f6342f7] Xorg_libX11_jll v1.8.13+0 [0c0b7dd1] Xorg_libXau_jll v1.0.13+0 [935fb764] Xorg_libXcursor_jll v1.2.4+0 [a3789734] Xorg_libXdmcp_jll v1.1.6+0 [1082639a] Xorg_libXext_jll v1.3.8+0 [d091e8ba] Xorg_libXfixes_jll v6.0.2+0 [a51aa0fd] Xorg_libXi_jll v1.8.3+0 [d1454406] Xorg_libXinerama_jll v1.1.7+0 [ec84b674] Xorg_libXrandr_jll v1.5.6+0 [ea2f1a96] Xorg_libXrender_jll v0.9.12+0 [a65dc6b1] Xorg_libpciaccess_jll v0.19.0+0 [c7cfdc94] Xorg_libxcb_jll v1.17.1+0 [cc61e674] Xorg_libxkbfile_jll v1.2.0+0 [e920d4aa] Xorg_xcb_util_cursor_jll v0.1.6+0 [12413925] Xorg_xcb_util_image_jll v0.4.1+0 [2def613f] Xorg_xcb_util_jll v0.4.1+0 [975044d2] Xorg_xcb_util_keysyms_jll v0.4.1+0 [0d47668e] Xorg_xcb_util_renderutil_jll v0.3.10+0 [c22f9ab0] Xorg_xcb_util_wm_jll v0.4.2+0 [35661453] Xorg_xkbcomp_jll v1.4.7+0 [33bec58e] Xorg_xkeyboard_config_jll v2.47.0+1 [c5fb5394] Xorg_xtrans_jll v1.6.0+0 [3161d3a3] Zstd_jll v1.5.7+1 [35ca27e7] eudev_jll v3.2.14+0 [214eeab7] fzf_jll v0.61.1+0 [a4ae2306] libaom_jll v3.13.3+0 [0ac62f75] libass_jll v0.17.4+0 [1183f4f0] libdecor_jll v0.2.2+0 [8e53e030] libdrm_jll v2.4.125+1 [2db6ffa8] libevdev_jll v1.13.4+0 [f638f0a6] libfdk_aac_jll v2.0.4+0 [36db933b] libinput_jll v1.28.1+0 [b53b4c65] libpng_jll v1.6.58+0 [9a156e7d] libva_jll v2.23.0+0 [f27f6e37] libvorbis_jll v1.3.8+0 [009596ad] mtdev_jll v1.1.7+0 ⌅ [1270edf5] x264_jll v10164.0.1+0 [dfaa095f] x265_jll v4.1.0+0 [d8fb68d0] xkbcommon_jll v1.13.0+0 [0dad84c5] ArgTools v1.1.2 [56f22d72] Artifacts v1.11.0 [2a0f44e3] Base64 v1.11.0 [ade2ca70] Dates v1.11.0 [f43a241f] Downloads v1.7.0 [7b1f6079] FileWatching v1.11.0 [b77e0a4c] InteractiveUtils v1.11.0 [ac6e5ff7] JuliaSyntaxHighlighting v1.12.0 [b27032c2] LibCURL v0.6.4 [76f85450] LibGit2 v1.11.0 [8f399da3] Libdl v1.11.0 [37e2e46d] LinearAlgebra v1.12.0 [56ddb016] Logging v1.11.0 [d6f4376e] Markdown v1.11.0 [a63ad114] Mmap v1.11.0 [ca575930] NetworkOptions v1.3.0 [44cfe95a] Pkg v1.12.1 [de0858da] Printf v1.11.0 [3fa0cd96] REPL v1.11.0 [9a3f8284] Random v1.11.0 [ea8e919c] SHA v0.7.0 [9e88b42a] Serialization v1.11.0 [6462fe0b] Sockets v1.11.0 [2f01184e] SparseArrays v1.12.0 [f489334b] StyledStrings v1.11.0 [fa267f1f] TOML v1.0.3 [a4e569a6] Tar v1.10.0 [8dfed614] Test v1.11.0 [cf7118a7] UUIDs v1.11.0 [4ec0a83e] Unicode v1.11.0 [e66e0078] CompilerSupportLibraries_jll v1.3.0+1 [deac9b47] LibCURL_jll v8.15.0+0 [e37daf67] LibGit2_jll v1.9.0+0 [29816b5a] LibSSH2_jll v1.11.3+1 [14a3606d] MozillaCACerts_jll v2025.11.4 [4536629a] OpenBLAS_jll v0.3.29+0 [05823500] OpenLibm_jll v0.8.7+0 [458c3c95] OpenSSL_jll v3.5.6+0 [efcefdf7] PCRE2_jll v10.44.0+1 [bea87d4a] SuiteSparse_jll v7.8.3+2 [83775a58] Zlib_jll v1.3.1+2 [8e850b90] libblastrampoline_jll v5.15.0+0 [8e850ede] nghttp2_jll v1.64.0+1 [3f19e933] p7zip_jll v17.7.0+0 Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. Testing Running tests... student missing variable: Test Failed at /home/pkgeval/.julia/packages/Grader/TPJcq/test/runtests.jl:153 Expression: (p.tests[1]).output == "\nUndefVarError: y not defined" Evaluated: "\nUndefVarError: `y` not defined in `Base.rsqmdxrytbsdvaqbqchs`\nSuggestion: check for spelling errors or missing imports." == "\nUndefVarError: y not defined" Stacktrace: [1] top-level scope @ ~/.julia/packages/Grader/TPJcq/test/runtests.jl:6 [2] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1777 [inlined] [3] macro expansion @ ~/.julia/packages/Grader/TPJcq/test/runtests.jl:42 [inlined] [4] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1777 [inlined] [5] macro expansion @ ~/.julia/packages/Grader/TPJcq/test/runtests.jl:141 [inlined] [6] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:1777 [inlined] [7] macro expansion @ ~/.julia/packages/Grader/TPJcq/test/runtests.jl:153 [inlined] [8] macro expansion @ /opt/julia/share/julia/stdlib/v1.12/Test/src/Test.jl:680 [inlined] Precompiling packages... 1463.3 ms ✓ Showoff 7789.5 ms ✓ StatsBase 1611.4 ms ✓ Graphite2_jll 1599.9 ms ✓ Libmount_jll 1141.5 ms ✓ EpollShim_jll 1492.1 ms ✓ LLVMOpenMP_jll 1529.9 ms ✓ Xorg_libICE_jll 1601.7 ms ✓ Xorg_libXau_jll 1584.0 ms ✓ libpng_jll 1643.5 ms ✓ libfdk_aac_jll 1617.7 ms ✓ libdrm_jll 1554.4 ms ✓ LAME_jll 1442.2 ms ✓ LERC_jll 1438.4 ms ✓ fzf_jll 1464.7 ms ✓ mtdev_jll 1505.5 ms ✓ Xorg_libXdmcp_jll 1497.9 ms ✓ x265_jll 1530.8 ms ✓ x264_jll 1483.4 ms ✓ libaom_jll 1526.0 ms ✓ libvorbis_jll 1542.6 ms ✓ Opus_jll 1341.1 ms ✓ Xorg_xtrans_jll 1469.3 ms ✓ libevdev_jll 1649.7 ms ✓ Libffi_jll 1461.9 ms ✓ eudev_jll 1479.5 ms ✓ FriBidi_jll 1458.6 ms ✓ Libuuid_jll 9587.9 ms ✓ ColorSchemes 1496.7 ms ✓ FreeType2_jll 1487.2 ms ✓ Dbus_jll 1586.1 ms ✓ GettextRuntime_jll 10846.4 ms ✓ Latexify 1550.7 ms ✓ Pixman_jll 1391.9 ms ✓ Xorg_libSM_jll 1583.5 ms ✓ Libtiff_jll 2581.6 ms ✓ JLFzf 1496.5 ms ✓ Xorg_libxcb_jll 1610.0 ms ✓ Wayland_jll 1645.6 ms ✓ libinput_jll 20741.1 ms ✓ PlotUtils 1599.4 ms ✓ Fontconfig_jll 1595.8 ms ✓ Glib_jll 2737.5 ms ✓ Latexify → SparseArraysExt 1489.5 ms ✓ Xorg_xcb_util_jll 1508.2 ms ✓ Xorg_libX11_jll 9807.7 ms ✓ PlotThemes 9846.8 ms ✓ RecipesPipeline 1601.9 ms ✓ Xorg_xcb_util_image_jll 1371.9 ms ✓ Xorg_xcb_util_keysyms_jll 1373.0 ms ✓ Xorg_xcb_util_renderutil_jll 1611.3 ms ✓ Xorg_xcb_util_wm_jll 1581.2 ms ✓ Xorg_libXrender_jll 1789.4 ms ✓ Xorg_libXext_jll 1515.1 ms ✓ Xorg_libXfixes_jll 1697.1 ms ✓ Xorg_libxkbfile_jll 1719.1 ms ✓ Xorg_xcb_util_cursor_jll 1715.9 ms ✓ Xorg_libXinerama_jll 1742.3 ms ✓ Xorg_libXrandr_jll 1820.9 ms ✓ Cairo_jll 1570.3 ms ✓ Libglvnd_jll 1746.5 ms ✓ Xorg_libXcursor_jll 1634.8 ms ✓ Xorg_libXi_jll 1609.5 ms ✓ libva_jll 1737.9 ms ✓ Xorg_xkbcomp_jll 1763.1 ms ✓ HarfBuzz_jll 1518.9 ms ✓ Xorg_xkeyboard_config_jll 1644.9 ms ✓ libass_jll 1643.8 ms ✓ Pango_jll 1763.0 ms ✓ xkbcommon_jll 1777.9 ms ✓ FFMPEG_jll 1510.2 ms ✓ Vulkan_Loader_jll 1796.3 ms ✓ libdecor_jll 1489.4 ms ✓ FFMPEG 1725.5 ms ✓ Qt6Base_jll 1772.5 ms ✓ GLFW_jll 1915.3 ms ✓ Qt6Svg_jll 1844.8 ms ✓ Qt6ShaderTools_jll 2022.1 ms ✓ GR_jll 1777.1 ms ✓ Qt6Declarative_jll 1769.4 ms ✓ Qt6Wayland_jll 7661.2 ms ✓ GR 127090.0 ms ✓ Plots 82 dependencies successfully precompiled in 325 seconds. 97 already precompiled. 20 dependencies precompiled but different versions are currently loaded (Base64, Dates, InteractiveUtils, JSON, JuliaSyntaxHighlighting, Logging, Markdown, Mmap, OrderedCollections, Parsers, PrecompileTools, Preferences, Printf, Serialization, Statistics, StyledStrings, TOML, Test, UUIDs and Unicode). Restart julia to access the new versions. Otherwise, 118 dependents of these packages may trigger further precompilation to work with the unexpected versions. Test Summary: | Pass Fail Total Time Grader.jl | 39 1 40 6m15.6s evalasmodule | 5 5 4.8s grade problem | 23 1 24 10.8s correct answer | 4 4 0.7s partially correct answer | 7 7 0.0s error in student code | 2 2 1.5s missing value in student code | 2 2 0.0s error in golden code | 2 2 0.0s student missing variable | 1 1 2 5.3s forbidden symbol | 2 2 1.9s type conversion error | 3 3 1.5s pl_JSON | 8 8 1.5s fill answers | 1 1 1.5s fill answers 2 | 1 1 0.7s plot | 1 1 5m56.2s RNG of the outermost testset: Random.Xoshiro(0xd7bd3128026ea4a4, 0x32445a267118295f, 0xd8a465f5c659df14, 0x2cb5749c69ea90f4, 0xe2b66daf1ae272d1) ERROR: LoadError: Some tests did not pass: 39 passed, 1 failed, 0 errored, 0 broken. in expression starting at /home/pkgeval/.julia/packages/Grader/TPJcq/test/runtests.jl:4 Testing failed after 388.67s ERROR: LoadError: Package Grader errored during testing Stacktrace: [1] pkgerror(msg::String) @ Pkg.Types /opt/julia/share/julia/stdlib/v1.12/Pkg/src/Types.jl:68 [2] test(ctx::Pkg.Types.Context, pkgs::Vector{PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool) @ Pkg.Operations /opt/julia/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:2538 [3] test @ /opt/julia/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:2387 [inlined] [4] test(ctx::Pkg.Types.Context, pkgs::Vector{PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::@Kwargs{io::IOContext{IO}}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:552 [5] test(pkgs::Vector{PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{julia_args::Cmd}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:169 [6] test(pkgs::Vector{String}; kwargs::@Kwargs{julia_args::Cmd}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:157 [7] test @ /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:157 [inlined] [8] #test#81 @ /opt/julia/share/julia/stdlib/v1.12/Pkg/src/API.jl:156 [inlined] [9] top-level scope @ /PkgEval.jl/scripts/evaluate.jl:223 [10] include(mod::Module, _path::String) @ Base ./Base.jl:306 [11] exec_options(opts::Base.JLOptions) @ Base ./client.jl:317 [12] _start() @ Base ./client.jl:550 in expression starting at /PkgEval.jl/scripts/evaluate.jl:214 PkgEval failed after 445.46s: package has test failures