wgsl-rs first milestone almost complete
I've been working hard on the first milestone of wgsl-rs, which is essentially to get it feature complete.
All WGSL vocabulary is now supported (WGSL "std" library including built-in functions, types, texture sampling),
excluding extensions.
The last step is including exhaustive roundtrip tests for builtins to ensure the CPU runtime is coherent with the GPU.
I've revamped the 2d renderer. It now has its own shaders, which are much better fit for purpose.
Next on the roadmap for the 2d renderer are events, hit testing, sprite sheet animations and 9-slice buttons.
wgsl-rs], significant work on [crabslab] adaptation
wgsl-rs]I've almost wrapped up the initial build out of [wgsl-rs].
The last bit is the derivative builtins (dpdx, dpdy, fwidth, etc.) and
the discard statement.
I really don't know what I'm going to do for the CPU semantics of the discard statement yet.
Maybe something like return Default::default()? That might work?
Anyway - I took a break from furiously pawing at my keyboard to produce code this weekend to furiously paw at my keyboard to write an article introducing the library.
Please check out the article introducing wgsl-rs.
crabslab]Both [crabslab] and [craballoc] are successfully using [wgsl-rs] in a branch.
Once [wgsl-rs] is on crates.io as a beta release, I'll do beta releases of those as well.
...
That's all the news that's fit to print :)
🙇☕
]]>The Memorandum of Understanding for the Renderling Ecosystem project has been signed with NLnet under the NGI0 Commons Fund. This makes the second year of NLnet funding official, and I'm excited to get started on the work ahead.
The biggest change in this round of work is that Renderling will be moving away from Rust-GPU and toward tighter integration with WebGPU and WGSL.
To support this, I've started work on wgsl-rs,
a procedural macro crate that lets you write WGSL shaders in a subset of Rust.
Code written with wgsl-rs runs on the CPU in Rust and on the GPU via
generated WGSL — same types, same logic, fully testable. It works on stable
Rust, produces human-readable WGSL output, needs no custom toolchains and even
generates wgpu linkage for you!
The funded milestones will take wgsl-rs to a beta release on crates.io, then
adapt crabslab and craballoc to work with it, and finally rewrite
Renderling's internals on the new stack.
podecsPart of the funded work is a new crate called podecs — a GPU-accelerated
Entity Component System where components are plain old data defined with
wgsl-rs, stored in GPU-accessible slabs, and systems run as compute shaders.
This is the piece that makes massively parallel scene management possible and
lays the groundwork for what comes next.
The second half of the project focuses on rendering. The plan is to build a GPU ray tracer first, validate it against a reference image suite, and then adapt it for real-time performance using techniques like wavefront path tracing and raymarching. The end goal is real-time global illumination integrated with Renderling's existing lighting system.
Rend3 has been an important renderer in the Rust ecosystem, and there are projects out there that depend on it. Part of this funded work includes designing and implementing an API compatibility layer for Rend3 users, along with a migration guide and example programs. If you have an existing Rend3 project, Renderling aims to be a viable path forward.
I'll be working through these milestones over the course of the year and posting updates here as things progress. If you're interested in contributing or just following along, check out the repos:
]]>crabslab changesI've managed to get a large PR finished on crabslab
that provides synchronization of values changed by shaders on the GPU back to
their CPU caches.
This change has been a long time coming, and it unlocks a lot of potential for interesting changes in Renderling, and beyond.
Specifically these changes help in any situation where a shader computes some data that then gets used on the CPU. And it does that with a minimal number of writes to and from the GPU.
This is all part of a grand scheme to blur the lines between GPU and CPU, and make GPU programming easier, which is essentially the main goal behind next year's worth of Renderling work.
Renderling was selected for another year of funding by NLnet. I'm currently working on the project plan and we have double the funding of last year, which means I'll be able to take on paid contributors and get more accomplished in the same amount of time.
Next year's work is focusing on the ecosystem and then riding that rising tide towards global illumination.
Global illumination is a very lofty goal, especially for a renderer that targets the web, but I think we can get there with some key tradeoffs.
]]>Part of this was fixing a bug in image based lighting regarding bindgroup invalidation.
]]>I've also made somewhat drastic website changes:
I'm moving away from the super long devlog format to something that hopefully is a bit more focused.
Now the devlog is broken up into two "things":
The devlog was getting huge, and it's a bit too chaotic. Now I'll be live-blogging feature development in specific articles, like I did for Light Tiling, Live. I'm hoping this does a better job of keeping the devlogs on-topic instead of being a big jumble of different things.
All other blurbs are news, so they can live on the news page. I'm just trying to stay organized as this project grows.
But - the old devlog will stay where it was, at devlog, for posterity and so the links still work.
The manual covers the basics, but lacks lighting examples.
Obviously it's a work in progress.
I'll be making sure that it's complete after getting it online. Even in its incomplete state, it has lots of workable, tested examples in it, and I hope it helps folks get up and running with the library.
There's a ton more documentation coverage. I did a big audit of the current documentation and added more where needed as well as revamped the existing docs.
The latest docs are now hosted here at /docs, due to the fact that Renderling
depends on a not yet released version of spirv-std, which the Rust-GPU group is still
working on releasing.
I've removed the crabslab::Id and craballoc::Hybrid* types from the public API.
I figured that it shouldn't be necessary for users to understand anything about slabs and descriptors.
Builder patterns
Now all the various resources (Primitive, Material, Vertices etc) adhere to a builder
pattern for configuration and updates.
Skybox is now separate from Ibl.
Up until this point, if you wanted to render a skybox, that skybox would also perform image based lighting automatically. I decided to decouple these now, as there are valid situations where you may not want IBL, but do want a skybox.
]]>What went wrong and what went right for Renderling in 2024. Written 9 Jan, 2025
o/ Hi y'all!
Welcome to the Renderling wrap article for 2024. I'm hoping to make writing this article a tradition.
This project started with its first commit in Sep 26, 2022 - so I've been working on this for roughly two years as a side project, after my day job and between raising two kids.
Without looking at the log of work, which is simply my list of closed PRs on GitHub, I can already tell you that I feel like I've accomplished a lot for Renderling this year!
This has been amazing, and has really changed my perspective on open source. The program has really been a guiding force for Renderling, and just knowing that people care about the outcome enough to invest in the project makes the overall quality of the software improve. Not to mention the fact that the money helps at a very basic level. I've already applied for 2025, 🤞. Thank you, nlnet!
And more specifically Lucien from Second Half ;)
This sponsorship spun out of the work I was doing to unblock Renderling's shaders.
You see, wgpu is a cross-platform graphics layer
that Renderling sits on top of. This layer abstracts over the popular graphics libraries like
DirectX, Vulkan, Metal, OpenGL, WebGL and WebGPU. It allows Renderling to target all platforms
without too much platform-specific code.
But it doesn't support all shader languages equally, and wgpu's SPIR-V support was lacking
some features - notably support for atomics.
Now this is important for Renderling because Renderling's shaders are all written in Rust, which then get compiled into SPIR-V, and atomics are an important building block when writing shaders. Oddly enough I've actually managed to avoid any use of atomics to this point, but that's changing very soon.
Anyway, long story short - Lucien saw that I was tackling atomic support in the SPIR-V frontend
of wgpu's shader translator and sponsored me to help fast-track that work, and I'm happy to
say that the work is done!
Thank you, Lucien!
And if you haven't checked out Second Half's game, "Meanwhile in Sector 80", you should! It looks amazing!
Other sponsorships
I also had a few other sponsorships, one on-going from my long-time collaborator and friend Zach, and a generous one-time donation from John Nagle, who is working on Sharpview, a metaverse viewer.
Also my buddy James Harton donated time on his machines for dedicated CI hardware.
Thank you, guys!
I've started fielding support questions on the Rust GameDev discord.
There have been a couple reddit posts, not by me, mostly by John Nagle, aka Animats.
GitHub stars have exploded this year:
Next year my social goals will be to get more and better documentation out there, with more examples.
I'd also like to pull in some PRs from other folks, and possibly find a guest maintainer. If 2025's nlnet grant goes through I'd like to contract some of the project's milestones out, as there's more work than I can manage myself, and having another person in the codebase would be good for organization, and clarity and focus in the API.
I'd also like to write some small games 😈.
Ok - let's enumerate the features and stuff added in 2024!
There were a lot of refactors and bug fixes, so I'm only going to mention the big rocks here.
Support for scene hierarchy through nodes that contain other nodes, where a "node"
is a rendering of some sort (a Renderlet in Renderling parlance)
A new bloom implementation based on learnopengl's guest article
rebuild of the animation system
Streamlined animation. Fixed some lingering bugs with rigging.
Support for multiple textures in the atlas. This greatly expanded the texturing capacity.
Support for animations using morph targets. These are important for facial animations, among other things.
Pre-rendering step that removes out-of-view geometry. Good speedup.
This would be a state of the art, two pass compute step to remove occluded geometry, but I put it aside during the debugging phase. I'll come back to this in 2025.
I put this here even though it's not exactly Renderling, but my shader compilation tools from
Renderling became the basis of this new, more general command line tool. Along with the work
of Thomas Buckley-Houston. We essentially mashed our compilation
tools together to come up with cargo-gpu. Check it out!
Last but not least I have to mention that this entire bespoke website was set up this year, and I wrote 4300 lines in the devlog! That's a lot of debugging. But I do tend to make lots of newlines...
Here's the renderling.xyz code if you care to take a look. It's very much made in a style that works for my workflow, and I'm often adding to it.
On my docket for 2025 are more features, documentation and examples.
I'd also love to work with an artist to get some good renderings to show off.
I guess I could always use some off-the-shelf assets, though.
Here's my feature short-list:
Just about everything I mentioned up to this point went right. I got a lot of features shipped in what I think is a pretty short amount of time.
But one glaring difficulty I had was with occlusion culling. This wasn't part of my nlnet grant but I dedicated a good chunk of time to it. I felt like I was always just a couple hours away from finishing, but the rabbit hole got deeper and deeper. I think sinking this much time bumped a few other milestones out of reach for me in 2024 - namely shadow mapping and light tiling. Next year I'll make it one of my milestones, if I win another grant. Then I can really focus on it without worry.
You can read about occlusion culling in the devlog here.
Another problem is that packing in all these features has fractured the API a bit. The project doesn't feel very "well planned" to me... ...because it isn't! I've been working on just getting things to work and haven't put any polish on it. The APIs need a bit of work around the edges to ensure the pieces line up well.
All in all, though, I'm very happy with this project :)
I'm going to keep this short and sweet, so that's it for 2024.
Thanks for reading and following along.
I wish you the best in 2025!
<3
]]>