needs to fix js failure: https://gitlab.haskell.org/ghc/ghc/-/jobs/2454388

the benchmark is conducted by running the command once to warnup the *.gbc files first, then running it the second time to collect the stats. having *.gbc files on disk means we skip bytecode generation, and using :print main instead of main skips actual execution of bytecode; the rationale of the mini benchmark is to focus on serialization and linking logic of bytecode, which have the lowest hanging fruits for optimization at the moment.

yes. i noticed a huge spike in cpu usage on my runner and hence this patch. i think the CPUS convention for our gitlab instance is underdocumented indeed, but it's better to document it elsewhere like ci-config, instead of a commit message in a specific repo that invokes gitlab ci.

I tried running the benchmark, but .gitlab/hello.hs doesn't typecheck with the given invocation. Could this affect the space usage?

ah, i forgot to mention, you need to add {-# LANGUAGE ImplicitPrelude #-} to .gitlab/hello.hs. if it doesn't typecheck this would of course significantly affect the stats! i'd be interested in whether you can reproduce similar improvements in your setup

this one is for the Binary class in binary package, which is transitively used by bytecode object serialization

needs to rebase and catch up with recent ghc-internal changes.

i agree that most inline stuff should just be static in headers and be left to the c compiler to decide whether to inline or not. but there are two issues with keeping -Winline:

• it's sensitive to the toolchain version, the c toolchain can make different decisions and emit different warnings based on what's inlined and not
• we also have vendored c sources like xxhash, and might vendor other cbits in the future, and we don't want to carry custom patches just for -Winline

i agree with opening a ticket and trying to remove some inline hints. i don't think -Winline should stay though.

i figured out the culprit, see #27066. it's my bad

This patch fixes missing +werror in validate flavour, which was an oversight in bb3a2ba1. Fixes #27066.

Cheng Shao (ec230141) at 18 Mar 20:43

it would be nice to move the custom client code to a proper test case, where a ghc api program invokes it as iserv and issues the custom iserv commands. but that can be left to subsequent patches.

Adds support to extend the external interpreter's server function with Custom commands.

For an example client, see the haskell-debugger's use of external commands https://github.com/well-typed/haskell-debugger/pull/222 to add features to the external interpreter such as listing threads or decoding the stack and serializing the result directly.

Closes #26652

would it make sense for CustomHandler to return Either String ByteString to provide a more informative error message for unhandled custom message?

I definitely won't die on this hill. If you feel strongly that I should change Word8 to Word32 I will.

me neither! as long as it's considered and you have your conclusion, then i trust it

thanks for the comments! do you have specific benchmarks where these 0/1/2 element special cases make a significant difference overall? for the record, the benchmark numbers shown in the previous thread is my mini benchmark of loading ghc as a library with breakpoints enabled by default, and i think it's a good approximation of the "average" workload involving a single home module large project.

i'm not convinced that preserving FlatBag in UnlinkedBCO makes much sense! in regard of compactness, the SmallArray fields are now unboxed, and each StgSmallMutArrPtrs closure only carries an extra word for length anyway. and it's not just about making UnlinkedBCO compact; using SmallArray uniformly allows us to reduce marshaling overhead in assembleBCO/linkBCO, and also slightly improves bytecode object serialization code paths, since there are no extra FlatBag variants to check at runtime.

What flavour did you run the benchmarks with?

quick-validate+debug_info

How does the heap profile look like with and without these changes?

i have not built with profiling enabled; the heap stats are generated by adding +RTS -s -RTS to ghci

Cheng Shao (f80dc8c7) at 18 Mar 17:41

needs to rebase again and catch up with recent ghc-internal changes.

Cheng Shao (c2c5a708) at 18 Mar 17:08

of course i want to enforce it in ci and stop playing whack a moles in ghc-internal, but even after i added 9.14 to bootstrap matrix, it passes in ci but still fails locally on my machine. very confusing.

the easiest way to enforce in ci is to add hadrian logic, but that's a very heavy hammer and i'd like a better understanding of why 9.14 validate job doesn't catch it first.