Move Struct marshaling to transient IL and adjust LayoutClass marshaling IL stub handling#125352
Move Struct marshaling to transient IL and adjust LayoutClass marshaling IL stub handling#125352jkoritzinsky wants to merge 25 commits intodotnet:mainfrom
Conversation
… as part of cleanup code.
…all new APIs for value type marshalling
…er stubs to the new transient IL/individual marshaling method stubs model and remove the old stubs.
…k around creation in managed code.
…ap allocated any more
|
Tagging subscribers to this area: @dotnet/interop-contrib |
There was a problem hiding this comment.
Pull request overview
This PR refactors CoreCLR struct and layout-class marshalling stub generation by moving struct marshalling to transient IL (per System.StubHelpers.StructureMarshaler<T> instantiation) and simplifying layout-class marshalling by generating dedicated IL stubs cached in managed statics (System.StubHelpers.LayoutClassMarshaler<T>), reducing reliance on the general IL stub cache and related runtime machinery.
Changes:
- Generate struct marshalling bodies via transient IL (
MethodDesc::TryGenerateTransientILImplementation) instead of cached IL stubs. - Add a new QCall to create layout-class marshal stubs on demand and cache them in managed code.
- Rework field-marshalling flag handling and remove struct-stub-specific caching/compile-time-state paths.
Reviewed changes
Copilot reviewed 26 out of 26 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/coreclr/vm/stubhelpers.h | Adds QCall declaration for creating layout-class marshal stubs. |
| src/coreclr/vm/stubhelpers.cpp | Implements StubHelpers_CreateLayoutClassMarshalStub QCall. |
| src/coreclr/vm/stubgen.h | Adjusts struct marshal stub argument indexing; adds transient IL generation hook. |
| src/coreclr/vm/qcallentrypoints.cpp | Removes old struct marshal stub QCall entry; adds new layout-class stub QCall entry. |
| src/coreclr/vm/prestub.cpp | Wires struct marshalling transient IL generation into prestub transient IL path. |
| src/coreclr/vm/olevariant.h | Removes managed-marshaler-code parameters from SAFEARRAY/record marshaling helpers. |
| src/coreclr/vm/olevariant.cpp | Updates SAFEARRAY/record marshaling paths to use new struct/layout marshalling entrypoints. |
| src/coreclr/vm/mlinfo.h | Reassigns marshal flag bits and adds new flags for struct marshalling scenarios; updates GenerateFieldIL signature. |
| src/coreclr/vm/mlinfo.cpp | Updates field IL generation to respect marshal flags and avoid invalid trailing NOPs. |
| src/coreclr/vm/metasig.h | Updates metasig definitions for updated marshaler entrypoints. |
| src/coreclr/vm/marshalnative.h | Replaces struct-stub QCall with MarshalNative_HasLayout QCall declaration. |
| src/coreclr/vm/marshalnative.cpp | Implements MarshalNative_HasLayout QCall. |
| src/coreclr/vm/ilstubresolver.h | Removes struct-marshal-specific loader-heap compile-time-state support. |
| src/coreclr/vm/ilstubresolver.cpp | Simplifies generated IL allocation and compile-time-state freeing logic. |
| src/coreclr/vm/ilstubcache.cpp | Removes struct marshal stub special-casing for keeping compile-time state alive. |
| src/coreclr/vm/ilmarshalers.h | Adds new marshal-flag helpers and updates field-marshalling emission paths. |
| src/coreclr/vm/ilmarshalers.cpp | Switches struct/layout marshalling call sites to new method-based entrypoints; updates array marshaller creation. |
| src/coreclr/vm/dllimport.h | Introduces MarshalOperation enum; replaces struct marshal stub APIs with new helpers. |
| src/coreclr/vm/dllimport.cpp | Implements transient IL generation for struct marshaling; adds layout-class stub generation; adds GetStructMarshallingMethod. |
| src/coreclr/vm/dispparammarshaler.cpp | Updates record/array marshaling to use new struct/layout marshalling method entrypoints. |
| src/coreclr/vm/dispatchinfo.cpp | Removes struct-stub lookup for SAFEARRAY element record marshaling. |
| src/coreclr/vm/corelib.h | Adds binder entries for StructureMarshaler<T> and LayoutClassMarshaler<T> methods; updates marshaler metasigs. |
| src/coreclr/tools/aot/ILCompiler.ReadyToRun/Compiler/ReadyToRunXmlRootProvider.cs | Adds null-check when instantiation isn’t possible for generic roots. |
| src/coreclr/System.Private.CoreLib/src/System/StubHelpers.cs | Adds StructureMarshaler<T> and LayoutClassMarshaler<T> managed helpers and managed caching for layout-class stubs. |
| src/coreclr/System.Private.CoreLib/src/System/Runtime/InteropServices/Marshal.CoreCLR.cs | Replaces struct-stub-based marshalling with layout probing + cached per-type marshal method entrypoints. |
| src/coreclr/System.Private.CoreLib/src/System/Runtime/CompilerServices/RuntimeHelpers.CoreCLR.cs | Minor formatting-only change (blank line). |
|
Would it be possible to do this change in two PRs? There are a lot of concerns here to try and reconcile in my head in a single PR. |
How would you like it split? I can try splitting it, but I'm not sure what split would end up making the code any easier to review. |
…e for boxed struct inputs (allows simplifying the use cases that have `object` inputs.
src/coreclr/System.Private.CoreLib/src/System/Runtime/InteropServices/Marshal.CoreCLR.cs
Show resolved
Hide resolved
de8b997 to
ace58eb
Compare
src/coreclr/System.Private.CoreLib/src/System/Runtime/InteropServices/Marshal.CoreCLR.cs
Show resolved
Hide resolved
…for the error logic as well (will throw during IL generation)
…oid leaking implementation details.
Co-authored-by: Copilot Autofix powered by AI <[email protected]>
…Inlining wrapper to ensure exception is handled when expected
Today, we emit IL for struct/layout class marshaling similar to how we handle P/Invoke, Delegate, and COM interop marshalling stubs: We emit an IL stub always, put it in the IL stub cache, etc.
However, struct marshalling stubs have some weird aspects to them today:
if-elsedecision tree (because our runtime IL emit tooling doesn't supportswitch) to jump to the right block.Marshal.StructureToPtr, so we had to add an additional faster caching layer inEEMarshalingData.This PR moves struct marshaling to be handled via Transient IL, with the marshaling logic presented as the IL for each instantiation of
System.StubHelpers.StructureMarshaler<T>.I wanted to do the same for layout-class marshaling, but it's not possible to do so because there's no way to opt-out of generic sharing. However, I was able to structure it similarly by having layout class marshalling continue to generate IL stubs. In the layout-class case, the stubs are no longer cached alongside the rest of the interop stubs with all of that machinery. Layout-class stubs are cached on the managed side in statics in
System.StubHelpers.LayoutClassMarshaler<T>, with a lock to ensure we don't double-generate the stubs.This allows us to remove a decent amount of logic around struct marshaling IL stubs and how they interact with P/Invoke stubs. Additionally, it paves a path towards moving P/Invoke IL bodies to use transient IL bodies instead of IL stubs for non-shared cases.