Package evaluation to test LLMCheatsheets on Julia 1.14.0-DEV.1260 (89243d1cdf*) started at 2025-11-16T16:33:29.948 ################################################################################ # Set-up # Installing PkgEval dependencies (TestEnv)... Set-up completed after 9.93s ################################################################################ # Installation # Installing LLMCheatsheets... Resolving package versions... Installed CodecZlib ─────────── v0.7.8 Installed ConcurrentUtilities ─ v2.5.0 Installed URIs ──────────────── v1.6.1 Installed Compat ────────────── v4.18.1 Installed ExceptionUnwrapping ─ v0.1.11 Installed ExprTools ─────────── v0.1.10 Installed AbstractTrees ─────── v0.4.5 Installed TranscodingStreams ── v0.11.3 Installed LoggingExtras ─────── v1.2.0 Installed OpenAI ────────────── v0.9.1 Installed BitFlags ──────────── v0.1.9 Installed OpenSSL ───────────── v1.6.0 Installed MbedTLS ───────────── v1.1.9 Installed JSON3 ─────────────── v1.14.3 Installed HTTP ──────────────── v1.10.19 Installed PrecompileTools ───── v1.3.3 Installed Parsers ───────────── v2.8.3 Installed LLMCheatsheets ────── v0.1.0 Installed JLLWrappers ───────── v1.7.1 Installed MbedTLS_jll ───────── v2.28.10+0 Installed SimpleBufferStream ── v1.2.0 Installed Preferences ───────── v1.5.0 Installed StructTypes ───────── v1.11.0 Installed Mocking ───────────── v0.8.1 Installed PromptingTools ────── v0.56.1 Installing 1 artifacts Installed artifact MbedTLS 2.2 MiB Updating `~/.julia/environments/v1.14/Project.toml` [bfa53fcd] + LLMCheatsheets v0.1.0 Updating `~/.julia/environments/v1.14/Manifest.toml` [1520ce14] + AbstractTrees v0.4.5 [d1d4a3ce] + BitFlags v0.1.9 [944b1d66] + CodecZlib v0.7.8 [34da2185] + Compat v4.18.1 [f0e56b4a] + ConcurrentUtilities v2.5.0 [460bff9d] + ExceptionUnwrapping v0.1.11 [e2ba6199] + ExprTools v0.1.10 [cd3eb016] + HTTP v1.10.19 [692b3bcd] + JLLWrappers v1.7.1 [0f8b85d8] + JSON3 v1.14.3 [bfa53fcd] + LLMCheatsheets v0.1.0 [e6f89c97] + LoggingExtras v1.2.0 [739be429] + MbedTLS v1.1.9 [78c3b35d] + Mocking v0.8.1 ⌅ [e9f21f70] + OpenAI v0.9.1 [4d8831e6] + OpenSSL v1.6.0 [69de0a69] + Parsers v2.8.3 [aea7be01] + PrecompileTools v1.3.3 [21216c6a] + Preferences v1.5.0 ⌅ [670122d1] + PromptingTools v0.56.1 [777ac1f9] + SimpleBufferStream v1.2.0 [856f2bd8] + StructTypes v1.11.0 [3bb67fe8] + TranscodingStreams v0.11.3 [5c2747f8] + URIs v1.6.1 [c8ffd9c3] + MbedTLS_jll v2.28.10+0 [0dad84c5] + ArgTools v1.1.2 [56f22d72] + Artifacts v1.11.0 [2a0f44e3] + Base64 v1.11.0 [ade2ca70] + Dates v1.11.0 [f43a241f] + Downloads v1.7.0 [7b1f6079] + FileWatching v1.11.0 [b77e0a4c] + InteractiveUtils v1.11.0 [ac6e5ff7] + JuliaSyntaxHighlighting v1.12.0 [b27032c2] + LibCURL v1.0.0 [76f85450] + LibGit2 v1.11.0 [8f399da3] + Libdl v1.11.0 [56ddb016] + Logging v1.11.0 [d6f4376e] + Markdown v1.11.0 [a63ad114] + Mmap v1.11.0 [ca575930] + NetworkOptions v1.3.0 [44cfe95a] + Pkg v1.13.0 [de0858da] + Printf v1.11.0 [3fa0cd96] + REPL v1.11.0 [9a3f8284] + Random v1.11.0 [ea8e919c] + SHA v1.0.0 [9e88b42a] + Serialization v1.11.0 [6462fe0b] + Sockets v1.11.0 [f489334b] + StyledStrings v1.11.0 [fa267f1f] + TOML v1.0.3 [a4e569a6] + Tar v1.10.0 [8dfed614] + Test v1.11.0 [cf7118a7] + UUIDs v1.11.0 [4ec0a83e] + Unicode v1.11.0 [e66e0078] + CompilerSupportLibraries_jll v1.3.0+1 [deac9b47] + LibCURL_jll v8.16.0+0 [e37daf67] + LibGit2_jll v1.9.1+0 [29816b5a] + LibSSH2_jll v1.11.3+1 [14a3606d] + MozillaCACerts_jll v2025.11.4 [458c3c95] + OpenSSL_jll v3.5.4+0 [efcefdf7] + PCRE2_jll v10.47.0+0 [83775a58] + Zlib_jll v1.3.1+2 [3161d3a3] + Zstd_jll v1.5.7+1 [8e850ede] + nghttp2_jll v1.68.0+1 [3f19e933] + p7zip_jll v17.7.0+0 Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated -m` Installation completed after 8.34s ################################################################################ # Precompilation # Precompiling PkgEval dependencies... Precompiling packages... WARNING: Imported binding Operations.project_rel_path was undeclared at import time during import to TestEnv. 4750.4 ms ✓ TestEnv 1 dependency successfully precompiled in 5 seconds. 27 already precompiled. 1 dependency had output during precompilation: ┌ TestEnv │ WARNING: Imported binding Operations.project_rel_path was undeclared at import time during import to TestEnv. └ ┌ Error: Failed to use TestEnv.jl; test dependencies will not be precompiled │ exception = │ UndefVarError: `project_rel_path` not defined in `TestEnv` │ Suggestion: this global was defined as `Pkg.Operations.project_rel_path` but not assigned a value. │ Stacktrace: │ [1] get_test_dir(ctx::Pkg.Types.Context, pkgspec::PackageSpec) │ @ TestEnv ~/.julia/packages/TestEnv/i9lgt/src/julia-1.11/common.jl:75 │ [2] test_dir_has_project_file │ @ ~/.julia/packages/TestEnv/i9lgt/src/julia-1.11/common.jl:52 [inlined] │ [3] maybe_gen_project_override! │ @ ~/.julia/packages/TestEnv/i9lgt/src/julia-1.11/common.jl:83 [inlined] │ [4] activate(pkg::String; allow_reresolve::Bool) │ @ TestEnv ~/.julia/packages/TestEnv/i9lgt/src/julia-1.11/activate_set.jl:12 │ [5] activate(pkg::String) │ @ TestEnv ~/.julia/packages/TestEnv/i9lgt/src/julia-1.11/activate_set.jl:9 │ [6] top-level scope │ @ /PkgEval.jl/scripts/precompile.jl:24 │ [7] include(mod::Module, _path::String) │ @ Base ./Base.jl:309 │ [8] exec_options(opts::Base.JLOptions) │ @ Base ./client.jl:344 │ [9] _start() │ @ Base ./client.jl:577 └ @ Main /PkgEval.jl/scripts/precompile.jl:26 Precompiling package dependencies... Precompiling packages... 1401.5 ms ✓ URIs 884.3 ms ✓ BitFlags 1152.6 ms ✓ AbstractTrees 1778.4 ms ✓ ConcurrentUtilities 847.2 ms ✓ ExprTools 1272.6 ms ✓ TranscodingStreams 848.3 ms ✓ SimpleBufferStream 1702.8 ms ✓ StructTypes 1097.8 ms ✓ LoggingExtras 1170.1 ms ✓ Compat 1158.4 ms ✓ Preferences 1672.3 ms ✓ ExceptionUnwrapping 4057.3 ms ✓ OpenSSL 975.1 ms ✓ CodecZlib 877.0 ms ✓ Compat → CompatLinearAlgebraExt 1234.8 ms ✓ JLLWrappers 888.5 ms ✓ PrecompileTools 1166.9 ms ✓ Mocking 1483.2 ms ✓ MbedTLS_jll 17528.2 ms ✓ Parsers 2099.0 ms ✓ MbedTLS 14189.1 ms ✓ JSON3 27100.0 ms ✓ HTTP 4470.8 ms ✓ OpenAI ┌ Warning: OPENAI_API_KEY variable not set! OpenAI models will not be available - set API key directly via `PromptingTools.OPENAI_API_KEY=`! └ @ PromptingTools ~/.julia/packages/PromptingTools/lqJyF/src/user_preferences.jl:165 36728.1 ms ✓ PromptingTools ┌ Warning: OPENAI_API_KEY variable not set! OpenAI models will not be available - set API key directly via `PromptingTools.OPENAI_API_KEY=`! └ @ PromptingTools ~/.julia/packages/PromptingTools/lqJyF/src/user_preferences.jl:165 14587.7 ms ✓ PromptingTools → MarkdownPromptingToolsExt ┌ Warning: OPENAI_API_KEY variable not set! OpenAI models will not be available - set API key directly via `PromptingTools.OPENAI_API_KEY=`! └ @ PromptingTools ~/.julia/packages/PromptingTools/lqJyF/src/user_preferences.jl:165 14674.1 ms ✓ LLMCheatsheets 27 dependencies successfully precompiled in 157 seconds. 33 already precompiled. 3 dependencies had output during precompilation: ┌ LLMCheatsheets │ ┌ Warning: OPENAI_API_KEY variable not set! OpenAI models will not be available - set API key directly via `PromptingTools.OPENAI_API_KEY=`! │ └ @ PromptingTools ~/.julia/packages/PromptingTools/lqJyF/src/user_preferences.jl:165 └ ┌ PromptingTools → MarkdownPromptingToolsExt │ ┌ Warning: OPENAI_API_KEY variable not set! OpenAI models will not be available - set API key directly via `PromptingTools.OPENAI_API_KEY=`! │ └ @ PromptingTools ~/.julia/packages/PromptingTools/lqJyF/src/user_preferences.jl:165 └ ┌ PromptingTools │ ┌ Warning: OPENAI_API_KEY variable not set! OpenAI models will not be available - set API key directly via `PromptingTools.OPENAI_API_KEY=`! │ └ @ PromptingTools ~/.julia/packages/PromptingTools/lqJyF/src/user_preferences.jl:165 └ Precompilation completed after 174.19s ################################################################################ # Testing # Testing LLMCheatsheets Status `/tmp/jl_3qQDl3/Project.toml` [4c88cf16] Aqua v0.8.14 [cd3eb016] HTTP v1.10.19 [0f8b85d8] JSON3 v1.14.3 [bfa53fcd] LLMCheatsheets v0.1.0 [78c3b35d] Mocking v0.8.1 ⌅ [670122d1] PromptingTools v0.56.1 [8dfed614] Test v1.11.0 Status `/tmp/jl_3qQDl3/Manifest.toml` [1520ce14] AbstractTrees v0.4.5 [4c88cf16] Aqua v0.8.14 [d1d4a3ce] BitFlags v0.1.9 [944b1d66] CodecZlib v0.7.8 [34da2185] Compat v4.18.1 [f0e56b4a] ConcurrentUtilities v2.5.0 [460bff9d] ExceptionUnwrapping v0.1.11 [e2ba6199] ExprTools v0.1.10 [cd3eb016] HTTP v1.10.19 [692b3bcd] JLLWrappers v1.7.1 [0f8b85d8] JSON3 v1.14.3 [bfa53fcd] LLMCheatsheets v0.1.0 [e6f89c97] LoggingExtras v1.2.0 [739be429] MbedTLS v1.1.9 [78c3b35d] Mocking v0.8.1 ⌅ [e9f21f70] OpenAI v0.9.1 [4d8831e6] OpenSSL v1.6.0 [69de0a69] Parsers v2.8.3 [aea7be01] PrecompileTools v1.3.3 [21216c6a] Preferences v1.5.0 ⌅ [670122d1] PromptingTools v0.56.1 [777ac1f9] SimpleBufferStream v1.2.0 [856f2bd8] StructTypes v1.11.0 [3bb67fe8] TranscodingStreams v0.11.3 [5c2747f8] URIs v1.6.1 [c8ffd9c3] MbedTLS_jll v2.28.10+0 [0dad84c5] ArgTools v1.1.2 [56f22d72] Artifacts v1.11.0 [2a0f44e3] Base64 v1.11.0 [ade2ca70] Dates v1.11.0 [f43a241f] Downloads v1.7.0 [7b1f6079] FileWatching v1.11.0 [b77e0a4c] InteractiveUtils v1.11.0 [ac6e5ff7] JuliaSyntaxHighlighting v1.12.0 [b27032c2] LibCURL v1.0.0 [76f85450] LibGit2 v1.11.0 [8f399da3] Libdl v1.11.0 [56ddb016] Logging v1.11.0 [d6f4376e] Markdown v1.11.0 [a63ad114] Mmap v1.11.0 [ca575930] NetworkOptions v1.3.0 [44cfe95a] Pkg v1.13.0 [de0858da] Printf v1.11.0 [3fa0cd96] REPL v1.11.0 [9a3f8284] Random v1.11.0 [ea8e919c] SHA v1.0.0 [9e88b42a] Serialization v1.11.0 [6462fe0b] Sockets v1.11.0 [f489334b] StyledStrings v1.11.0 [fa267f1f] TOML v1.0.3 [a4e569a6] Tar v1.10.0 [8dfed614] Test v1.11.0 [cf7118a7] UUIDs v1.11.0 [4ec0a83e] Unicode v1.11.0 [e66e0078] CompilerSupportLibraries_jll v1.3.0+1 [deac9b47] LibCURL_jll v8.16.0+0 [e37daf67] LibGit2_jll v1.9.1+0 [29816b5a] LibSSH2_jll v1.11.3+1 [14a3606d] MozillaCACerts_jll v2025.11.4 [458c3c95] OpenSSL_jll v3.5.4+0 [efcefdf7] PCRE2_jll v10.47.0+0 [83775a58] Zlib_jll v1.3.1+2 [3161d3a3] Zstd_jll v1.5.7+1 [8e850ede] nghttp2_jll v1.68.0+1 [3f19e933] p7zip_jll v17.7.0+0 Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. Testing Running tests... ┌ Warning: OPENAI_API_KEY variable not set! OpenAI models will not be available - set API key directly via `PromptingTools.OPENAI_API_KEY=`! └ @ PromptingTools ~/.julia/packages/PromptingTools/lqJyF/src/user_preferences.jl:165 ┌ Warning: OPENAI_API_KEY variable not set! OpenAI models will not be available - set API key directly via `PromptingTools.OPENAI_API_KEY=`! └ @ PromptingTools ~/.julia/packages/PromptingTools/lqJyF/src/user_preferences.jl:165 ┌ Warning: OPENAI_API_KEY variable not set! OpenAI models will not be available - set API key directly via `PromptingTools.OPENAI_API_KEY=`! └ @ PromptingTools ~/.julia/packages/PromptingTools/lqJyF/src/user_preferences.jl:165 [ Info: Found 1 files and 0 folders in README [ Info: Found 0 files and 0 folders in [ Info: Scanning all files in octocat/Hello-World [ Info: Will scan the following paths: README and file types: [ Info: Found 1 files and 0 folders in README [ Info: Scanned 1 files in 0.0 seconds. Test Summary: | Pass Total Time LLMCheatsheets.jl | 35 35 2m57.5s Testing LLMCheatsheets tests passed Testing completed after 206.49s PkgEval succeeded after 445.1s