Skip to content

Hitting OOM in LLVM for the wasi-wasm aot smoke tests #96630

New issue
New issue
Closed
Closed
Hitting OOM in LLVM for the wasi-wasm aot smoke tests#96630
Labels
Known Build ErrorUse this to report build issues in the .NET Helix tabarch-wasmWebAssembly architecturearea-Codegen-AOT-monoblocking-clean-ciBlocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'os-wasiRelated to WASI variant of arch-wasm
@tannergooding

Description

@tannergooding
tannergooding
opened on Jan 8, 2024

From #96455:

C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : Failed to compile C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.bc -> C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.o [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : C:\helix\work\workitem\e\publish>C:\Windows\System32\chcp.com 65001 1>nul  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : C:\helix\work\workitem\e\publish>setlocal [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : C:\helix\work\workitem\e\publish>set errorlevel=dummy  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : C:\helix\work\workitem\e\publish>set errorlevel=  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : C:\helix\work\workitem\e\publish>C:\helix\work\correlation\build\\wasi-sdk\\bin\clang.exe "@C:\helix\work\correlation\build\microsoft.netcore.app.runtime.wasi-wasm\runtimes\wasi-wasm\native\src\wasi-default.rsp" "@C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\wasi-compile-bc.rsp" -c -o "C:\Users\ContainerAdministrator\AppData\Local\Temp\tmp2z1yzq.tmp" "C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.bc"  [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : clang version 16.0.0 [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : Target: wasm32-unknown-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : Thread model: posix [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : InstalledDir: C:/helix/work/correlation/build//wasi-sdk//bin [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error :  (in-process) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error :  "C:/helix/work/correlation/build/wasi-sdk/bin/clang.exe" -cc1 -triple wasm32-unknown-wasi -emit-obj -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name aot-instances.dll.bc -mrelocation-model static -mframe-pointer=none -ffp-contract=on -fno-rounding-math -mconstructor-aliases -target-cpu generic -fvisibility=hidden -mllvm -treat-scalable-fixed-error-as-warning -debug-info-kind=constructor -dwarf-version=4 -debugger-tuning=gdb -v -fcoverage-compilation-dir=C:/helix/work/workitem/e/publish -resource-dir C:/helix/work/correlation/build/wasi-sdk/lib/clang/16 -Oz -fdebug-compilation-dir=C:/helix/work/workitem/e/publish -ferror-limit 19 -fgnuc-version=4.2.1 -vectorize-slp -o "C:\\Users\\ContainerAdministrator\\AppData\\Local\\Temp\\tmp2z1yzq.tmp" -x ir "C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\aot-instances.dll.bc" [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : clang -cc1 version 16.0.0 based upon LLVM 16.0.0 default target wasm32-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : LLVM ERROR: out of memory [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : Allocation failed [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script. [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : Stack dump: [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0.	Program arguments: C:\\helix\\work\\correlation\\build\\\\wasi-sdk\\\\bin\\clang.exe @C:\\helix\\work\\correlation\\build\\microsoft.netcore.app.runtime.wasi-wasm\\runtimes\\wasi-wasm\\native\\src\\wasi-default.rsp @C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\wasi-compile-bc.rsp -c -o C:\\Users\\ContainerAdministrator\\AppData\\Local\\Temp\\tmp2z1yzq.tmp C:\\helix\\work\\workitem\\e\\wasm_build\\obj\\wasm\\for-build\\aot-instances.dll.bc [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 1.	Code generation [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 2.	Running pass 'Function Pass Manager' on module 'C:\helix\work\workitem\e\wasm_build\obj\wasm\for-build\aot-instances.dll.bc'. [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 3.	Running pass 'WebAssembly Assembly Printer' on function '@aot_instances_aot_wrapper_gsharedvt_out_sig_void_this_cl1e_Mono_dValueTuple_601_3clong_3e_u4' [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : Exception Code: 0xC000001D [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x003EFB73 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x6CFCC902 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x003D23A1 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x003D90B0 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x003D931E <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x6CF5DAAA <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x00525EA8 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x0029FF6F <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x0029B927 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x575D4010 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x7759D96D <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x6CF596BB <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : 0x0029DE57 <unknown module> [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : clang: error: clang frontend command failed due to signal (use -v to see invocation) [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : clang version 16.0.0 [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : Target: wasm32-unknown-wasi [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : Thread model: posix [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : InstalledDir: C:/helix/work/correlation/build//wasi-sdk//bin [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]
C:\helix\work\correlation\build\wasm-shared\WasmApp.Common.targets(819,5): error : clang: note: diagnostic msg: Error generating preprocessed source(s) - no preprocessable inputs. [took 129.44s] [C:\helix\work\workitem\e\publish\ProxyProjectForAOTOnHelix.proj]

This is possibly related to the use of static virtuals in interfaces + generics + vectors in the VectorMath class here (latest commit is working around the issue by not using generics, so look back one commit to see): https://github.com/dotnet/runtime/pull/96455/files#diff-b19d582dbc1021ce89fed7edc91778f2b8ca5087d777c552d536aed34f1625e5

CI link: https://github.com/dotnet/runtime/runs/20216910077

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=544187

{
  "ErrorMessage": "error : LLVM ERROR: out of memory",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=544187
Error message validated: error : LLVM ERROR: out of memory
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 1/29/2024 6:38:16 PM UTC

Report

Build Definition Test Pull Request
544686 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #95402
544678 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #96895
544646 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97216
544782 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97660
544045 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97604
544662 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #93619
544814 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97651
544810 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97361
544798 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97643
544772 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97658
544575 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97640
544717 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97654
544681 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #95543
544692 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97052
544677 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97589
544670 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #93953
544190 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97632
544304 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97508
544584 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97559
544550 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #86674
544478 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97644
544354 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97388
544295 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97637
544240 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #91317
544273 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #96910
544187 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97441
544167 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97559
544105 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #96910
544091 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #97340
544037 dotnet/runtime System.Collections.Specialized.Tests.WorkItemExecution #91317

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 22 30

Metadata

Metadata

Assignees

No one assigned

    Labels

    Known Build ErrorUse this to report build issues in the .NET Helix tabarch-wasmWebAssembly architecturearea-Codegen-AOT-monoblocking-clean-ciBlocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms'os-wasiRelated to WASI variant of arch-wasm

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions