Package evaluation to test CommonMark on Julia 1.10.10 (c8be17dcfd*) started at 2026-02-02T14:14:39.731 ################################################################################ # Set-up # Installing PkgEval dependencies (TestEnv)... Activating project at `~/.julia/environments/v1.10` Set-up completed after 5.15s ################################################################################ # Installation # Installing CommonMark... Resolving package versions... Updating `~/.julia/environments/v1.10/Project.toml` [a80b9123] + CommonMark v0.10.2 Updating `~/.julia/environments/v1.10/Manifest.toml` [a80b9123] + CommonMark v0.10.2 ⌅ [aea7be01] + PrecompileTools v1.2.1 [21216c6a] + Preferences v1.5.1 [ade2ca70] + Dates [de0858da] + Printf [fa267f1f] + TOML v1.0.3 [4ec0a83e] + Unicode Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m` Installation completed after 8.79s ################################################################################ # Precompilation # Precompiling PkgEval dependencies... Precompiling package dependencies... Precompiling packages... 11953.1 ms ✓ CommonMark 877.5 ms ✓ pandoc_jll 1056.8 ms ✓ CommonMark → CommonMarkMarkdownExt 1127.2 ms ✓ CommonMark → CommonMarkMarkdownASTExt 4 dependencies successfully precompiled in 20 seconds. 45 already precompiled. Precompilation completed after 29.69s ################################################################################ # Testing # Testing CommonMark Status `/tmp/jl_XtMlUf/Project.toml` [a80b9123] CommonMark v0.10.2 [682c06a0] JSON v1.4.0 [d0879d2d] MarkdownAST v0.1.2 ⌅ [aea7be01] PrecompileTools v1.2.1 [324d217c] ReferenceTests v0.10.6 [f8b46487] TestItemRunner v1.1.4 [ddb6d928] YAML v0.4.16 [c5432543] pandoc_jll v3.6.4+0 [d6f4376e] Markdown [44cfe95a] Pkg v1.10.0 [8dfed614] Test Status `/tmp/jl_XtMlUf/Manifest.toml` [1520ce14] AbstractTrees v0.4.5 [3da002f7] ColorTypes v0.12.1 [c3611d14] ColorVectorSpace v0.11.0 [5ae59095] Colors v0.13.1 [a80b9123] CommonMark v0.10.2 [a8cc5b0e] Crayons v4.1.1 [ab62b9b5] DeepDiffs v1.2.0 [b4f34e82] Distances v0.10.12 [5789e2e9] FileIO v1.18.0 [53c48c17] FixedPointNumbers v0.8.5 [c817782e] ImageBase v0.1.7 [a09fc81d] ImageCore v0.10.5 [9b13fd28] IndirectArrays v1.0.0 [692b3bcd] JLLWrappers v1.7.1 [682c06a0] JSON v1.4.0 [dbb5928d] MappedArrays v0.4.3 [d0879d2d] MarkdownAST v0.1.2 [e94cdb99] MosaicViews v0.3.4 [6fe1bfb0] OffsetArrays v1.17.0 [5432bcbf] PaddedViews v0.5.12 [69de0a69] Parsers v2.8.3 ⌅ [aea7be01] PrecompileTools v1.2.1 [21216c6a] Preferences v1.5.1 [189a3867] Reexport v1.2.2 [324d217c] ReferenceTests v0.10.6 [ae029012] Requires v1.3.1 [45858cf5] Sixel v0.1.5 [cae243ae] StackViews v0.1.2 [82ae8749] StatsAPI v1.8.0 [69024149] StringEncodings v0.3.7 [ec057cc2] StructUtils v2.6.2 [62fd8b95] TensorCore v0.1.1 [f8b46487] TestItemRunner v1.1.4 [1c621080] TestItems v1.0.0 [c8c2cc18] XTermColors v0.2.1 [ddb6d928] YAML v0.4.16 [aacddb02] JpegTurbo_jll v3.1.4+0 [94ce4f54] Libiconv_jll v1.18.0+0 [b53b4c65] libpng_jll v1.6.54+0 [075b6546] libsixel_jll v1.10.5+0 [c5432543] pandoc_jll v3.6.4+0 [0dad84c5] ArgTools v1.1.1 [56f22d72] Artifacts [2a0f44e3] Base64 [ade2ca70] Dates [f43a241f] Downloads v1.6.0 [7b1f6079] FileWatching [b77e0a4c] InteractiveUtils [b27032c2] LibCURL v0.6.4 [76f85450] LibGit2 [8f399da3] Libdl [37e2e46d] LinearAlgebra [56ddb016] Logging [d6f4376e] Markdown [ca575930] NetworkOptions v1.2.0 [44cfe95a] Pkg v1.10.0 [de0858da] Printf [3fa0cd96] REPL [9a3f8284] Random [ea8e919c] SHA v0.7.0 [9e88b42a] Serialization [6462fe0b] Sockets [2f01184e] SparseArrays v1.10.0 [10745b16] Statistics v1.10.0 [fa267f1f] TOML v1.0.3 [a4e569a6] Tar v1.10.0 [8dfed614] Test [cf7118a7] UUIDs [4ec0a83e] Unicode [e66e0078] CompilerSupportLibraries_jll v1.1.1+0 [deac9b47] LibCURL_jll v8.4.0+0 [e37daf67] LibGit2_jll v1.6.4+0 [29816b5a] LibSSH2_jll v1.11.0+1 [c8ffd9c3] MbedTLS_jll v2.28.1010+0 [14a3606d] MozillaCACerts_jll v2025.12.2 [4536629a] OpenBLAS_jll v0.3.23+5 [bea87d4a] SuiteSparse_jll v7.2.1+1 [83775a58] Zlib_jll v1.2.13+1 [8e850b90] libblastrampoline_jll v5.11.0+0 [8e850ede] nghttp2_jll v1.52.0+1 [3f19e933] p7zip_jll v17.4.0+2 Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. Testing Running tests... ┌ Info: Reference Test for "inline_math.txt" failed: │ - DIFF ------------------------ │ """ │ - Some \e[35mmath\e[39m. │ + Some math. │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/math/inline_math.txt" └ actual = "/tmp/jl_fjSsTF/inline_math.txt" math: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/math.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Nothing, transform::Nothing) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] (::Main.var"##225".Utilities.var"#1#3"{Main.var"##225".Utilities.var"#1#2#4"{String}})(base_name::String, ast::CommonMark.Node, reference_dir::String) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:8 [10] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/math.jl:13 [11] eval @ ./boot.jl:385 [inlined] [12] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [13] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [14] #invokelatest#2 @ ./essentials.jl:892 [inlined] [15] invokelatest @ ./essentials.jl:889 [inlined] [16] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [17] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [18] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [19] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [20] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [21] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [22] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [23] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [24] include(fname::String) @ Base.MainInclude ./client.jl:487 [25] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/math.jl:13 ┌ Info: Reference Test for "internal_pipes.txt" failed: │ - DIFF ------------------------ │ """ │ ┏━━━┯━━━━┯━━━┓ │ - ┃ 1 │ 10 │ \e[36m|\e[39m ┃ │ + ┃ 1 │ 10 │ | ┃ │ ┠───┼────┼───┨ │ - ┃ \e[3m|\e[23m │ \e[32m|\e[39m │ ┃ │ + ┃ | │ | │ ┃ │ ┃ 1 │ 2 │ 3 ┃ │ ┗━━━┷━━━━┷━━━┛ │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/tables/internal_pipes.txt" └ actual = "/tmp/jl_Zmso7c/internal_pipes.txt" tables: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/tables.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Nothing, transform::Nothing) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] (::Main.var"##225".Utilities.var"#1#3"{Main.var"##225".Utilities.var"#1#2#4"{String}})(base_name::String, ast::CommonMark.Node, reference_dir::String) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:8 [10] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/tables.jl:52 [11] eval @ ./boot.jl:385 [inlined] [12] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [13] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [14] #invokelatest#2 @ ./essentials.jl:892 [inlined] [15] invokelatest @ ./essentials.jl:889 [inlined] [16] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [17] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [18] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [19] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [20] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [21] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [22] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [23] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [24] include(fname::String) @ Base.MainInclude ./client.jl:487 [25] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/tables.jl:52 ┌ Info: Reference Test for "multiple_extensions.txt" failed: │ - DIFF ------------------------ │ """ │ - \e[34;1m#\e[39;22m Integration Tests │ + # Integration Tests │ │ - \e[31;1m┌ Danger ─────────────────────────────────────────────────────────────────────\e[39;22m │ - \e[31;1m│\e[39;22m Warning (\e[31m@cite\e[39m) admonition\e[31m[^1]\e[39m… │ - \e[31;1m│\e[39;22m  │ - \e[31;1m│\e[39;22m \e[35m│\e[39m \e[90mmaths\e[39m │ - \e[31;1m│\e[39;22m  │ - \e[31;1m│\e[39;22m \e[31m┌ [^1] ─────────────────────────────────────────────────────────────────────\e[39m │ - \e[31;1m│\e[39;22m \e[31m│\e[39m 1 │ - \e[31;1m│\e[39;22m \e[31m│\e[39m  │ - \e[31;1m│\e[39;22m \e[31m│\e[39m footnote content │ - \e[31;1m│\e[39;22m \e[31m└───────────────────────────────────────────────────────────────────────────\e[39m │ - \e[31;1m└─────────────────────────────────────────────────────────────────────────────\e[39;22m │ + ┌ Danger ───────────────────────────────────────────────────────────────────── │ + │ Warning (@cite) admonition[^1]… │ + │  │ + │ │ maths │ + │  │ + │ ┌ [^1] ───────────────────────────────────────────────────────────────────── │ + │ │ 1 │ + │ │  │ + │ │ footnote content │ + │ └─────────────────────────────────────────────────────────────────────────── │ + └───────────────────────────────────────────────────────────────────────────── │ │ - \e[33;1m┌ Warning ────────────────────────────────────────────────────────────────────\e[39;22m │ - \e[33;1m│\e[39;22m “Warning” (\e[31m@cite\e[39m) admonition\e[31m[^2]\e[39m. │ - \e[33;1m│\e[39;22m  │ - \e[33;1m│\e[39;22m ┏━━━━━━━━━┓ │ - \e[33;1m│\e[39;22m ┃ table ┃ │ - \e[33;1m│\e[39;22m ┠─────────┨ │ - \e[33;1m│\e[39;22m ┃ content ┃ │ - \e[33;1m│\e[39;22m ┗━━━━━━━━━┛ │ - \e[33;1m│\e[39;22m \e[31m┌ [^2] ─────────────────────────────────────────────────────────────────────\e[39m │ - \e[33;1m│\e[39;22m \e[31m│\e[39m 2 │ - \e[33;1m│\e[39;22m \e[31m└───────────────────────────────────────────────────────────────────────────\e[39m │ - \e[33;1m└─────────────────────────────────────────────────────────────────────────────\e[39;22m │ + ┌ Warning ──────────────────────────────────────────────────────────────────── │ + │ “Warning” (@cite) admonition[^2]. │ + │  │ + │ ┏━━━━━━━━━┓ │ + │ ┃ table ┃ │ + │ ┠─────────┨ │ + │ ┃ content ┃ │ + │ ┗━━━━━━━━━┛ │ + │ ┌ [^2] ───────────────────────────────────────────────────────────────────── │ + │ │ 2 │ + │ └─────────────────────────────────────────────────────────────────────────── │ + └───────────────────────────────────────────────────────────────────────────── │ │ - \e[36;1m┌ Info ───────────────────────────────────────────────────────────────────────\e[39;22m │ - \e[36;1m│\e[39;22m ‘Tip’ (\e[31m@cite\e[39m) ‘admonition’\e[31m[^3]\e[39m. │ - \e[36;1m│\e[39;22m  │ - \e[36;1m│\e[39;22m \e[31m┌ [^3] ─────────────────────────────────────────────────────────────────────\e[39m │ - \e[36;1m│\e[39;22m \e[31m│\e[39m \e[35mx\e[39m │ - \e[36;1m│\e[39;22m \e[31m└───────────────────────────────────────────────────────────────────────────\e[39m │ - \e[36;1m└─────────────────────────────────────────────────────────────────────────────\e[39;22m │ + ┌ Info ─────────────────────────────────────────────────────────────────────── │ + │ ‘Tip’ (@cite) ‘admonition’[^3]. │ + │  │ + │ ┌ [^3] ───────────────────────────────────────────────────────────────────── │ + │ │ x │ + │ └─────────────────────────────────────────────────────────────────────────── │ + └───────────────────────────────────────────────────────────────────────────── │ │ - \e[36;1m┌ Note ───────────────────────────────────────────────────────────────────────\e[39;22m │ - \e[36;1m│\e[39;22m Note (\e[31m@cite\e[39m) “admonition”\e[31m[^4]\e[39m. │ - \e[36;1m│\e[39;22m  │ - \e[36;1m│\e[39;22m \e[90mlatex\e[39m │ - \e[36;1m│\e[39;22m  │ - \e[36;1m│\e[39;22m \e[31m┌ [^4] ─────────────────────────────────────────────────────────────────────\e[39m │ - \e[36;1m│\e[39;22m \e[31m│\e[39m 4 │ - \e[36;1m│\e[39;22m \e[31m└───────────────────────────────────────────────────────────────────────────\e[39m │ - \e[36;1m└─────────────────────────────────────────────────────────────────────────────\e[39;22m │ + ┌ Note ─────────────────────────────────────────────────────────────────────── │ + │ Note (@cite) “admonition”[^4]. │ + │  │ + │ latex │ + │  │ + │ ┌ [^4] ───────────────────────────────────────────────────────────────────── │ + │ │ 4 │ + │ └─────────────────────────────────────────────────────────────────────────── │ + └───────────────────────────────────────────────────────────────────────────── │ │ - \e[32;1m┌ Tip ────────────────────────────────────────────────────────────────────────\e[39;22m │ - \e[32;1m│\e[39;22m Tip (\e[31m@cite\e[39m) admonition\e[31m[^5]\e[39m. │ - \e[32;1m│\e[39;22m  │ - \e[32;1m│\e[39;22m \e[31m┌ [^5] ─────────────────────────────────────────────────────────────────────\e[39m │ - \e[32;1m│\e[39;22m \e[31m│\e[39m 5 │ - \e[32;1m│\e[39;22m \e[31m└───────────────────────────────────────────────────────────────────────────\e[39m │ - \e[32;1m└─────────────────────────────────────────────────────────────────────────────\e[39;22m │ + ┌ Tip ──────────────────────────────────────────────────────────────────────── │ + │ Tip (@cite) admonition[^5]. │ + │  │ + │ ┌ [^5] ───────────────────────────────────────────────────────────────────── │ + │ │ 5 │ + │ └─────────────────────────────────────────────────────────────────────────── │ + └───────────────────────────────────────────────────────────────────────────── │ │ - \e[34;1m##\e[39;22m References │ + ## References │ │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/references/integration/multiple_extensions.txt" └ actual = "/tmp/jl_NhrDd6/multiple_extensions.txt" multiple_extensions: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/integration.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Nothing, transform::Nothing) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] (::Main.var"##225".Utilities.var"#1#3"{Main.var"##225".Utilities.var"#1#2#4"{String}})(base_name::String, ast::CommonMark.Node, reference_dir::String) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:8 [10] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/integration.jl:25 [11] eval @ ./boot.jl:385 [inlined] [12] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [13] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [14] #invokelatest#2 @ ./essentials.jl:892 [inlined] [15] invokelatest @ ./essentials.jl:889 [inlined] [16] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [17] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [18] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [19] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [20] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [21] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [22] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [23] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [24] include(fname::String) @ Base.MainInclude ./client.jl:487 [25] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/integration.jl:25 ┌ Info: Reference Test for "multiple_refs_same_label.txt" failed: │ - DIFF ------------------------ │ """ │ - A \e[34;4mcool ref\e[39;24m and \e[34;4mhere\e[39;24m. │ + A cool ref and here. │ │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/referencelinks/multiple_refs_same_label.txt" └ actual = "/tmp/jl_fis0YG/multiple_refs_same_label.txt" referencelinks: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/referencelinks.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Nothing, transform::Nothing) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] (::Main.var"##225".Utilities.var"#1#3"{Main.var"##225".Utilities.var"#1#2#4"{String}})(base_name::String, ast::CommonMark.Node, reference_dir::String) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:8 [10] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/referencelinks.jl:185 [11] eval @ ./boot.jl:385 [inlined] [12] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [13] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [14] #invokelatest#2 @ ./essentials.jl:892 [inlined] [15] invokelatest @ ./essentials.jl:889 [inlined] [16] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [17] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [18] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [19] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [20] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [21] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [22] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [23] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [24] include(fname::String) @ Base.MainInclude ./client.jl:487 [25] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/referencelinks.jl:185 ┌ Info: Reference Test for "link.txt" failed: │ - DIFF ------------------------ │ """ │ - \e[34;4mlink\e[39;24m │ + link │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/smartlinks/link.txt" └ actual = "/tmp/jl_iCrPhY/link.txt" smart_links: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/smartlinks.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Dict{String, Any}, transform::Function) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/smartlinks.jl:35 [10] eval @ ./boot.jl:385 [inlined] [11] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [12] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [13] #invokelatest#2 @ ./essentials.jl:892 [inlined] [14] invokelatest @ ./essentials.jl:889 [inlined] [15] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [16] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [17] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [18] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [19] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [20] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [21] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [22] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [23] include(fname::String) @ Base.MainInclude ./client.jl:487 [24] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/smartlinks.jl:35 ┌ Info: Reference Test for "note_basic.txt" failed: │ - DIFF ------------------------ │ """ │ - \e[34;1m┌ Note ───────────────────────────────────────────────────────────────────────\e[39;22m │ - \e[34;1m│\e[39;22m Basic note content. │ - \e[34;1m└─────────────────────────────────────────────────────────────────────────────\e[39;22m │ + ┌ Note ─────────────────────────────────────────────────────────────────────── │ + │ Basic note content. │ + └───────────────────────────────────────────────────────────────────────────── │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/github_alerts/note_basic.txt" └ actual = "/tmp/jl_wNEMdP/note_basic.txt" github_alerts: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/github_alerts.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Nothing, transform::Nothing) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] (::Main.var"##225".Utilities.var"#1#3"{Main.var"##225".Utilities.var"#1#2#4"{String}})(base_name::String, ast::CommonMark.Node, reference_dir::String) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:8 [10] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/github_alerts.jl:95 [11] eval @ ./boot.jl:385 [inlined] [12] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [13] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [14] #invokelatest#2 @ ./essentials.jl:892 [inlined] [15] invokelatest @ ./essentials.jl:889 [inlined] [16] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [17] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [18] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [19] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [20] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [21] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [22] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [23] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [24] include(fname::String) @ Base.MainInclude ./client.jl:487 [25] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/github_alerts.jl:95 ┌ Info: Reference Test for "basic.txt" failed: │ - DIFF ------------------------ │ """ │ - ‘some’ \e[33minterpolation\e[39m \e[33minterpolation\e[39m │ + ‘some’ interpolation interpolation │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/interpolation/basic.txt" └ actual = "/tmp/jl_PHSVYz/basic.txt" interpolation: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/interpolation.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Nothing, transform::Nothing) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] (::Main.var"##225".Utilities.var"#1#3"{Main.var"##225".Utilities.var"#1#2#4"{String}})(base_name::String, ast::CommonMark.Node, reference_dir::String) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:8 [10] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/interpolation.jl:23 [11] eval @ ./boot.jl:385 [inlined] [12] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [13] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [14] #invokelatest#2 @ ./essentials.jl:892 [inlined] [15] invokelatest @ ./essentials.jl:889 [inlined] [16] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [17] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [18] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [19] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [20] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [21] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [22] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [23] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [24] include(fname::String) @ Base.MainInclude ./client.jl:487 [25] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/interpolation.jl:23 ┌ Info: Reference Test for "unbracketed_unknown.txt" failed: │ - DIFF ------------------------ │ """ │ - \e[31m@unknown\e[39m │ + @unknown │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/citations/unbracketed_unknown.txt" └ actual = "/tmp/jl_5CzWZO/unbracketed_unknown.txt" citations: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/citations.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Dict{String, Any}, transform::Nothing) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] test(bib::Vector{Any}, ast::CommonMark.Node, base_name::String) @ Main.var"##259" ~/.julia/packages/CommonMark/puSYy/test/extensions/citations.jl:13 [10] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/citations.jl:19 [11] eval @ ./boot.jl:385 [inlined] [12] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [13] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [14] #invokelatest#2 @ ./essentials.jl:892 [inlined] [15] invokelatest @ ./essentials.jl:889 [inlined] [16] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [17] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [18] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [19] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [20] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [21] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [22] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [23] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [24] include(fname::String) @ Base.MainInclude ./client.jl:487 [25] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/citations.jl:19 ┌ Info: Reference Test for "code.txt" failed: │ - DIFF ------------------------ │ """ │ - \e[36mcode\e[39m │ + code │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/writers/references/term/code.txt" └ actual = "/tmp/jl_zU65Zp/code.txt" terminal_writer: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/writers/term.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference @ ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [inlined] [6] _test_single_format(test_dir::String, filename::String, text::String, parser::CommonMark.Parser, format_func::typeof(CommonMark.term)) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:62 [7] (::Main.var"##225".Utilities.var"#6#7"{String, CommonMark.Parser})(filename::String, text::String, format::Function) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:47 [8] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/writers/term.jl:10 [9] eval @ ./boot.jl:385 [inlined] [10] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [11] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [12] #invokelatest#2 @ ./essentials.jl:892 [inlined] [13] invokelatest @ ./essentials.jl:889 [inlined] [14] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [15] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [16] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [17] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [18] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [19] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [20] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [21] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [22] include(fname::String) @ Base.MainInclude ./client.jl:487 [23] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/writers/term.jl:10 ┌ Info: Reference Test for "inline_raw.txt" failed: │ - DIFF ------------------------ │ """ │ - \e[90mhtml\e[39m\e[90mlatex\e[39m\e[90mtypst\e[39m │ + htmllatextypst │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/raw/inline_raw.txt" └ actual = "/tmp/jl_feetlV/inline_raw.txt" raw_content: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/raw.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Nothing, transform::Nothing) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] (::Main.var"##225".Utilities.var"#1#3"{Main.var"##225".Utilities.var"#1#2#4"{String}})(base_name::String, ast::CommonMark.Node, reference_dir::String) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:8 [10] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/raw.jl:13 [11] eval @ ./boot.jl:385 [inlined] [12] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [13] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [14] #invokelatest#2 @ ./essentials.jl:892 [inlined] [15] invokelatest @ ./essentials.jl:889 [inlined] [16] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [17] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [18] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [19] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [20] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [21] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [22] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [23] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [24] include(fname::String) @ Base.MainInclude ./client.jl:487 [25] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/raw.jl:13 ┌ Info: Reference Test for "custom_highlighter.txt" failed: │ - DIFF ------------------------ │ """ │ - \e[36m│\e[39m \e[90mNO TERM HIGHLIGHTING\e[39m │ + │ NO TERM HIGHLIGHTING │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/highlights/custom_highlighter.txt" └ actual = "/tmp/jl_tc8f1i/custom_highlighter.txt" highlights: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/highlights.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Nothing, transform::Function) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/highlights.jl:41 [10] eval @ ./boot.jl:385 [inlined] [11] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [12] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [13] #invokelatest#2 @ ./essentials.jl:892 [inlined] [14] invokelatest @ ./essentials.jl:889 [inlined] [15] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [16] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [17] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [18] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [19] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [20] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [21] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [22] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [23] include(fname::String) @ Base.MainInclude ./client.jl:487 [24] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/highlights.jl:41 ┌ Info: Reference Test for "basic_warning.txt" failed: │ - DIFF ------------------------ │ """ │ - \e[39;1m┌ warning ────────────────────────────────────────────────────────────────────\e[22m │ - \e[39;1m│\e[22m This is a warning. │ - \e[39;1m└─────────────────────────────────────────────────────────────────────────────\e[22m │ + ┌ warning ──────────────────────────────────────────────────────────────────── │ + │ This is a warning. │ + └───────────────────────────────────────────────────────────────────────────── │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/fenceddivs/basic_warning.txt" └ actual = "/tmp/jl_HyDPPS/basic_warning.txt" fenceddivs: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/fenceddivs.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Nothing, transform::Nothing) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] (::Main.var"##225".Utilities.var"#1#3"{Main.var"##225".Utilities.var"#1#2#4"{String}})(base_name::String, ast::CommonMark.Node, reference_dir::String) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:8 [10] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/fenceddivs.jl:75 [11] eval @ ./boot.jl:385 [inlined] [12] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [13] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [14] #invokelatest#2 @ ./essentials.jl:892 [inlined] [15] invokelatest @ ./essentials.jl:889 [inlined] [16] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [17] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [18] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [19] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [20] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [21] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [22] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [23] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [24] include(fname::String) @ Base.MainInclude ./client.jl:487 [25] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/fenceddivs.jl:75 ┌ Info: Reference Test for "warning_basic.txt" failed: │ - DIFF ------------------------ │ """ │ - \e[33;1m┌ Warning ────────────────────────────────────────────────────────────────────\e[39;22m │ - \e[33;1m│\e[39;22m text │ - \e[33;1m└─────────────────────────────────────────────────────────────────────────────\e[39;22m │ + ┌ Warning ──────────────────────────────────────────────────────────────────── │ + │ text │ + └───────────────────────────────────────────────────────────────────────────── │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/admonitions/warning_basic.txt" └ actual = "/tmp/jl_JyVFq2/warning_basic.txt" admonitions: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/admonitions.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Nothing, transform::Nothing) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] (::Main.var"##225".Utilities.var"#1#3"{Main.var"##225".Utilities.var"#1#2#4"{String}})(base_name::String, ast::CommonMark.Node, reference_dir::String) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:8 [10] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/admonitions.jl:10 [11] eval @ ./boot.jl:385 [inlined] [12] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [13] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [14] #invokelatest#2 @ ./essentials.jl:892 [inlined] [15] invokelatest @ ./essentials.jl:889 [inlined] [16] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [17] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [18] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [19] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [20] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [21] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [22] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [23] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [24] include(fname::String) @ Base.MainInclude ./client.jl:487 [25] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/admonitions.jl:10 ┌ Info: Reference Test for "link_only.txt" failed: │ - DIFF ------------------------ │ """ │ - text\e[31m[^1]\e[39m │ + text[^1] │ """ │ ------------------------------- │ reference = "/home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/references/footnotes/link_only.txt" └ actual = "/tmp/jl_geXHqG/link_only.txt" footnotes: Error During Test at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/footnotes.jl:2 Got exception outside of a @test LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")', or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE` to "true" and re-run the tests via Pkg. Stacktrace: [1] error(s::String) @ Base ./error.jl:35 [2] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing; kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:165 [3] test_reference(reference_file::FileIO.File{FileIO.DataFormat{:TXT}, String}, raw_actual::Text{String}, equiv::Nothing, rendermode::Nothing) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:113 [4] test_reference(filename::String, raw_actual::Text{String}; by::Nothing, render::Nothing, format::Nothing, kw::@Kwargs{}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:110 [5] test_reference(filename::String, raw_actual::Text{String}) @ ReferenceTests ~/.julia/packages/ReferenceTests/QAKab/src/test_reference.jl:105 [6] test_all_formats(test_dir::String, base_name::String, ast::CommonMark.Node, reference_dir::String; formats::Vector{Symbol}, env::Nothing, transform::Nothing) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:41 [7] test_all_formats @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:14 [inlined] [8] #1#2 @ ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:9 [inlined] [9] (::Main.var"##225".Utilities.var"#1#3"{Main.var"##225".Utilities.var"#1#2#4"{String}})(base_name::String, ast::CommonMark.Node, reference_dir::String) @ Main.var"##225".Utilities ~/.julia/packages/CommonMark/puSYy/test/test_utils.jl:8 [10] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/extensions/footnotes.jl:10 [11] eval @ ./boot.jl:385 [inlined] [12] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String) @ Base ./loading.jl:2149 [13] include_string(m::Module, txt::String, fname::String) @ Base ./loading.jl:2159 [14] #invokelatest#2 @ ./essentials.jl:892 [inlined] [15] invokelatest @ ./essentials.jl:889 [inlined] [16] #8 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:119 [inlined] [17] withpath(f::TestItemRunner.var"#8#12"{String, String, Module}, path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/vendored_code.jl:7 [18] #7 @ ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:118 [inlined] [19] cd(f::TestItemRunner.var"#7#11"{String, String, Module}, dir::String) @ Base.Filesystem ./file.jl:112 [20] run_testitem(filepath::String, use_default_usings::Bool, setups::Vector{Symbol}, package_name::String, original_code::String, line::Int64, column::Int64, test_setup_module_set::TestItemRunner.TestSetupModuleSet, testsetups::Dict{Symbol, Any}) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:117 [21] run_tests(path::String; filter::Nothing, verbose::Bool) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:230 [22] run_tests(path::String) @ TestItemRunner ~/.julia/packages/TestItemRunner/ax5HJ/src/TestItemRunner.jl:134 [23] top-level scope @ ~/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 [24] include(fname::String) @ Base.MainInclude ./client.jl:487 [25] top-level scope @ none:6 in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/extensions/footnotes.jl:10 Test Summary: | Pass Error Total Time Package | 1786 14 1800 1m58.2s test/ambiguities.jl | 1 1 2.1s test/extensions/typography.jl | 5 5 18.6s test/sourcepos.jl | 6 6 4.7s test/writers/latex.jl | 23 23 2.7s test/extensions/subscript.jl | 20 20 0.6s test/writers/html.jl | 25 25 0.2s test/ast.jl | 9 9 0.2s test/stdlib_compat.jl | 56 56 3.3s test/readers/markdownast.jl | 82 82 13.5s test/extensions/math.jl | 3 1 4 3.6s math | 3 1 4 3.6s test/extensions/frontmatter.jl | 14 14 15.6s test/extensions/autoidentifiers.jl | 16 16 0.9s test/extensions/tasklist.jl | 20 20 0.5s test/extensions/strikethrough.jl | 15 15 0.1s test/parsing.jl | 75 75 5.6s test/extensions/tables.jl | 19 1 20 2.6s tables | 19 1 20 2.6s test/extensions/mark.jl | 22 22 0.4s test/integration.jl | 3 1 4 3.0s multiple_extensions | 3 1 4 3.0s test/extensions/referencelinks.jl | 73 1 74 3.1s referencelinks | 73 1 74 3.1s test/spec_compliance.jl | 655 655 7.0s test/extensions/smartlinks.jl | 3 1 4 1.9s smart_links | 3 1 4 1.9s test/extensions/github_alerts.jl | 27 1 28 0.6s github_alerts | 27 1 28 0.6s test/writers/notebook.jl | 18 18 0.3s test/extensions/interpolation.jl | 13 1 14 0.4s interpolation | 13 1 14 0.4s test/samples.jl | 26 26 0.5s test/extensions/superscript.jl | 25 25 0.2s test/transform.jl | 21 21 3.3s test/builders.jl | 101 101 3.0s test/spec.jl | 2 2 0.1s test/extensions/attributes.jl | 94 94 0.6s test/roundtrip.jl | 5 5 2.5s test/extensions/citations.jl | 3 1 4 1.0s citations | 3 1 4 1.0s test/writers/term.jl | 1 1 0.2s terminal_writer | 1 1 0.2s test/unicodes.jl | 10 10 0.0s test/extensions/delimiter_interactions.jl | 26 26 0.1s test/extensions/raw.jl | 3 1 4 0.4s raw_content | 3 1 4 0.4s test/writers/json.jl | 46 46 5.0s test/extensions/highlights.jl | 2 1 3 0.9s highlights | 2 1 3 0.9s test/extensions/fenceddivs.jl | 21 1 22 0.4s fenceddivs | 21 1 22 0.4s test/writers/typst.jl | 23 23 0.0s test/extensions/admonitions.jl | 3 1 4 0.2s admonitions | 3 1 4 0.2s test/readers/stdlib.jl | 53 53 2.3s test/extensions/footnotes.jl | 3 1 4 0.2s footnotes | 3 1 4 0.2s test/writers/markdown.jl | 71 71 0.1s test/readers/json.jl | 45 45 5.6s ERROR: LoadError: Some tests did not pass: 1786 passed, 0 failed, 14 errored, 0 broken. in expression starting at /home/pkgeval/.julia/packages/CommonMark/puSYy/test/runtests.jl:2 Testing failed after 147.5s ERROR: LoadError: Package CommonMark errored during testing Stacktrace: [1] pkgerror(msg::String) @ Pkg.Types /opt/julia/share/julia/stdlib/v1.10/Pkg/src/Types.jl:70 [2] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool) @ Pkg.Operations /opt/julia/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:2034 [3] test @ /opt/julia/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1915 [inlined] [4] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::@Kwargs{io::Base.PipeEndpoint}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.10/Pkg/src/API.jl:444 [5] test(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.PipeEndpoint, kwargs::@Kwargs{julia_args::Cmd}) @ Pkg.API /opt/julia/share/julia/stdlib/v1.10/Pkg/src/API.jl:159 [6] test @ /opt/julia/share/julia/stdlib/v1.10/Pkg/src/API.jl:147 [inlined] [7] #test#74 @ /opt/julia/share/julia/stdlib/v1.10/Pkg/src/API.jl:146 [inlined] [8] top-level scope @ /PkgEval.jl/scripts/evaluate.jl:223 in expression starting at /PkgEval.jl/scripts/evaluate.jl:214 PkgEval failed after 202.77s: package tests unexpectedly errored