Project Robius in 2024: one year of Rust App Dev

2025-01-08T00:00:00+00:00

Author: Kevin Boos</a>. Last updated January 16th, 2025.</em> Discuss this on Reddit</a>.</p>

This past year marked the first year of work on Project Robius</strong>: an open-source decentralized endeavor to enable developers to write immersive, fully-featured apps in pure 🦀 Rust 🦀 that work seamlessly across all major platforms.</p>

ℹ️ Project Robius in a nutshell</strong> </font></p>
While GUIs are the foundation of an app, there is more to developing a modern featureful app than just drawing a UI. Project Robius aims to fill in the gaps in the Rust app dev ecosystem by focusing on everything except</em> the UI, such as abstractions for platform features & OS services, build & packaging tooling, and more. We leave UI work to the experts behind the many excellent Rust UI toolkits under active development.</p> </blockquote>
In this post, we'll take a look back on what we've accomplished so far to make the world of Rust App Dev a little better:</p>

The crates we've published for accessing platform-provided features from Rust code</li>
The major apps we've built using Makepad</a> + Robius together</li>
The contributions we've made to existing open-source projects in the App Dev space</li>
The connections we've fostered throughout the Rust community</li> </ol>
We'll also take a deeper look at Robrix</strong>, a multi-platform Matrix</a> chat client written from scratch in Rust using the Makepad UI toolkit</a> and Robius components.</p>
1. Robius crates for platform feature abstractions</h2>
As our primary objective, we have published several crates intended to be used directly by app devs to access a given platform feature or OS service from their app. The main goal here is for each crate to offer a safe, platform-agnostic abstraction, such that the app dev need not worry about writing any platform-specific code or dealing with each platforms' idiosyncracies.</p>
We began working on these crates in late Spring of 2024. While development started out gradually, we are significantly ramping up our efforts for the coming year and expect to publish more frequently. That being said, here is the current list:</p>

robius-location</code></a>: access the current geolocation of the user's device</li>
robius-authentication</code></a>: display a native biometric or password authentication prompt</li>
robius-open</code></a>: open a URI or file in a different app (determined by the system)</li>
robius-directories</code></a>: access platform-standard directory locations for app data, user data, config, cache, etc a fork of the directories</code></a> crate that adds support for Android</li> </ul> </li> robius-url-handler</code></a>: register your Rust app as the default handler for a URL scheme or file association</li> robius-keychain</code></a>: store and retrieve secure data/passwords from the platform's secure storage facility This crate was written from scratch, but we discovered keyring-rs</code></a> shortly after finishing it. We intend to contribute our additional features, mostly Android support, into keyring-rs</code>.</li> </ul> </li> robius-dialog</code></a>: (WIP) display a native dialog for showing a message or allowing the user to pick a file, directory, image, etc. This offers a custom implementation for iOS and Android, but uses rfd</code></a> under the hood for Desktop platforms.</li> </ul> </li> </ul> We've also released some "lower-level" crates that aren't intended for direct use by an app developer, but they'd be useful for other developers that want to create their own platform abstractions. The above crates depend on these in various ways.</p> android-build</code></a>: enables a Rust crate to automatically build Java code for Android targets, as part of a cargo build process via a build.rs</code> build script. The Java classfile(s) can then be used by your Rust app, typically via one of the above platform feature abstraction crates.</li> </ul> </li> robius-android-env</code></a>: abstracts access to Android states owned by various UI toolkits. Many Android platform features require passing in the current activity or accessing the JavaVM or JNI environment state.</li> This crate enables us to write other Rust crates that access Android platform features, such that they work seamlessly across many different UI toolkits.</li> Directly supports ndk-context</code></a>, an existing crate which is compatible with many other UI toolkit components, e.g., [Winit] This enables all of the above platform feature abstraction crates (plus any crate that depends on robius-android-env</code>) to work with Winit-based apps on Android.</li> </ul> </li> Compared to ndk-context</code>, the robius-android-env</code> crate offers a "batteries-included" experience that automatically "just works" with supported UI toolkits, such that the app dev doesn't have to add any code to make things work.</li> </ul> </li> </ul> 2. Apps built in 2024 using Makepad + Robius</h2> We (with help from many collaborators) have built both small proof-of-concept demo apps and larger "flagship" apps using Makepad + Robius. Two of the most complex flagship apps we've been developing in 2024 are:</p> Robrix</a>: a Matrix chat client for power users</li> Moly</a>: a local LLM chat runtime and AI agent explorer (previously "Moxin")</li> </ul> Both of these apps are fully open-source and have releases available on their GitHub pages linked above, in case you'd like to download and try them out. Note that it's best to build them from source for the most up-to-date experience.</p> Robrix: an up-and-coming Matrix chat client for power users</h3> We started Robrix</a> about one year ago with the intention of it being a "flagship" Robius app — one that would help drive the development (and priority) of various Robius components and demonstrate their utility. Since then, our plans for Robrix have expanded beyond it serving as just a demo app or a basic Matrix client; we discuss our longer-term, multi-stage vision for Robrix in the next post</a>.</p> Robrix has come a long way over the past year, thanks to 750+ commits from 10 contributors! Since starting from scratch, we have created a functional Matrix chat client with most fundamental features already complete and working well, as shown by our feature status tracker below.</p> </a> With these features in place, we have began dogfooding Robrix as a daily Matrix client!</p> While not all main features are complete, Robrix does</em> already have some cool features that help both power users and casual users be more productive. The biggest unique feature of Robrix is an "IDE-like" desktop UI that can display multiple rooms side-by-side in separate tabs, which can be docked and moved around via drag-n-drop actions. No more wasted horizontal space!</p> </a> Another cool feature is that Robrix's UI can automatically transition to different view layouts based on window size. This enables our single codebase to run seamlessly on desktop and mobile platforms, but you can also use any view on any platform if you want. For example, we frequently enjoy using the mid-size tablet view (below, left) or the narrow mobile view (below, middle) on a smaller laptop screen too, in addition to on our smartphones (below, right).</p> </a> </a> </a> </div> Beyond a sleek UI, Robrix also leverages multiple Robius crates for deep integration with the native platform:</p> robius-open</code> to open URLs, images, and downloaded files</li> robius-location</code> to obtain and share the user's current location in a Matrix room</li> robius-url-handler</code> to register Robrix as a default handler for the matrix:</code> URL scheme (and others)</li> robius-directories</code> to ensure that we store app data and cached content in the platform-canonical directories</li> robius-keychain</code> to store a user's login session tokens (this is a WIP)</li> robius-packaging-commands</code> to help easily build app bundles for desktop platforms using cargo-packager</li> In the future, we'll allow users to mark individual rooms as "secret", such that they are hidden behind an authentication prompt provided by robius-authentication</code></li> </ul> In addition to a sleek UI and robust platform integration, Robrix achieves high performance and efficiency thanks to its underlying pure-Rust stack and Makepad's emphasis on lightweight, performant code. Robrix consistently hits the maximum 120 FPS on an older M1 Macbook Pro, remaining smooth and responsive even when scrolling through 10+ rooms displayed side-by-side. We achieve this while using only around 26-30% of the system RAM that major Electron-based Matrix desktop clients consume to display a single room. (Note: these are preliminary figures that require deeper benchmarking analysis before drawing conclusions from them.)</p> Most importantly, thanks to the power of Makepad and Robius, Robrix has zero platform-specific code. This makes it easy to maintain and develop features/bugfixes quickly, as you don't have to consider the idiosyncracies of each platform. Thus, we invite you to check out our codebase and contribute any cool features that you'd love to have!</p> To learn more about Robrix, check out the following resources:</p> Robrix's GitHub repository</a></li> A recent conference talk about Robrix</a> (PDF slides</a>)</li> Robrix's Project Tracker on GitHub</a></li> Chat with us about Robrix on Matrix</a></li> </ul> Moly: chat with local LLMs and custom AI agents</h3> Moly</a> (f.k.a. Moxin</em>) is a pure Rust GUI client for running local Large Language Models (LLMs) and chatting with various AI agents. You can discover, browse, and download major open-source AI models:</p> </a> and then chat with them locally</em> without contacting any hosted LLM service.</p> </a> Like Robrix, Moly was started about one year ago completely from scratch, and has been a significant driver for the development of fundamental Makepad widgets, components, and Robius infrastructure. For example, Project Robius contributions to Moly and to other projects (at the request of Moly) were driven by these needs:</p> Better packaging logic and build configuration, which became robius-packaging-commands</code></a>. This cooperates with cargo-packager</code> to generate Moly app bundles for all 3 major desktop platforms.</li> </ul> </li> Portable Rust installation & setup "scripts" that run before the GUI app starts, which became moly-runner</code></a>. This was needed to install and configure the WasmEdge WASM runtime</a>, which is how Moly runs LLMs locally.</li> This is also useful for setting up the complex WasmEdge + Moly development environment in just one click.</li> </ul> </li> Many new Makepad widgets: modals, pop-up notifications, sliding panels, draggable sliders, etc.</li> Standardized app behaviors to be more platform-compliant and canonical, e.g., proper use of app data directories.</li> </ul> To learn more about Moly, check out this blog post</a> that demonstrates more cool features, screenshots, and examples of what you can do with Moly. Due to constraints from the underlying WasmEdge runtime, Moly currently runs only on major desktop platforms (Linux, macOS, Windows), but support for iOS and Android is planned.</p> 3. Select contributions to other Rust app dev projects</h2> In addition to creating, maintaining, and publishing our own crates for Rust app dev, we also strive to contribute to and improve existing crates that are already prominently used in the ecosystem.</p> We began using and making contributions to cargo-packager</code></a>, a packaging solution for Rust apps on Desktop target platforms created and open-sourced by Crab-Nebula, the folks behind the excellent Tauri ecosystem Our contributions</a> were mostly minor bugfixes and improvements to allow the packaging infrastructure to be configured more flexibly</li> As previously mentioned, we published robius-packaging-commands</code></a>, a companion to cargo-packager</code> that makes it easier to build & configure complex apps Automatically calculates the set of dependencies for Debian .deb</code> packages</li> Automatically handles Makepad configuration and resource/asset discovery & bundling</li> </ul> </li> We intend to add support for other Desktop package formats, namely Flatpack</li> We also plan to contribute support for generating mobile app bundles, namely Android</li> </ul> </li> We have made myriad major contributions</a> to the Makepad UI toolkit, as Robrix and Moly are two of the most complex/demanding apps built in Makepad Improvements to PortalList</code>, a virtual viewport list with infinite scrolling Better API with more introspection into the positional & visibility state of items in the list, its scrolling state, and its item caching behavior</li> Efficient implementations of smooth scrolling animations, e.g., jump to bottom or jump to a given item index</li> Redesign how items are stored and indexed, and how visible items are tracked</li> </ul> </li> Rich text formatting for displaying both HTML and Markdown content Including support for most formatting-relevant tags: (un)ordered lists, strikethrough/underline, coloring, indentation, blockquote, code, etc.</li> Special handling of interactive components like HTML links, which must preserve external formatting</li> </ul> </li> Multiple new widgets: avatar images with text fallback, abstractions over rich (HTML) text and plaintext, modals, sliding panes, etc</li> Make writing event handlers more ergonomic by avoiding mutable borrows when querying views/widgets</li> Redesign of underlying Android platform layer to allow external crates to access Android system states</li> Enable correct discovery of resource/asset files in macOS/iOS app bundles</li> Many improvements to cargo-makepad</code>, a build tool to generate mobile app packages Overhaul code to generate Android APKs</li> Properly install/configure the NDK toolchain on all 3 desktop platforms, plus enable building native code (via cc-rs</code>)</li> Ensure backwards compatibility with standard Android Studio-managed SDKs</li> </ul> </li> An improved app lifecycle model with dedicated events for all lifecycle stages, which is consistent across all platforms</li> Easier and more ergonomic Actions</code> (widget-to-widget message events) Plus support for delivering an action to a widget from a background thread or async task context</li> </ul> </li> </ul> </li> We made minor contributions</a> to the security-framework</code></a> crate, which offers Rust bindings to Apple's security framework (for TLS, keychain, etc) We added a few missing APIs to enabling updating or deleting keychain items, which we needed to fully implement robius-keychain</code></a></li> </ul> </li> We implemented a Rust auto-installer and configurer for the WasmEdge WASM runtime</a>, as mentioned above</a> This massively simplifies both the developer-side build process and the user installation procedure for Moly, which relies on WasmEdge to run LLMs locally.</li> We hope to transform this into the official install script for WasmEdge and upstream it for general usage there, as much of the effort involved was devoted to extracting the precise system configuration required to select and install the proper WasmEdge release.</li> </ul> </li> </ul> 4. Cross-collaboration with other UI and App Dev orgs</h2> Beyond publishing crates and developing apps, we also want to bring together people of all stripes across the Ruist UI and App Dev ecosystem. To that end, Project Robius hosted an App Dev unconference</a> at RustNL 2024 (and also GOSIM Beijing 2024), in which a few dozen Rust developers from across the world met up to discuss the shared problems we all face in developing Rust apps and UI toolkits. We discussed everything from build tooling to text layout, accessibility, Winit compatibility, and more. A few of the topics & ideas from the unconference(s) have already made it past the discussion phase and have become real projects!</p> kittest</code></a>: a universal UI testing framework built upon the AccessKit</a> accessibility framework, spearheaded by the eGUI team!</li> Dioxus's work on hotreloading</a> not just UI DSL code, but even real Rust code that implements app behavior!</li> Feedback given to the Rust project teams, primarily lang, libs, and compiler. We focused on changes to Rust that will make future Rust apps easier to write, with simplified and more ergonomic code patterns for async and more.</li> </ul> </li> </ul> In addition, thanks to our colleague Sid Askary, we began monthly meet-ups to chat about ongoing Rust UI & App Dev concerns, and to share ideas, solutions, progress updates. Attendees vary, but often include teammembers from Robius</a>, Makepad</a>, the Linebender organization</a> (behind Xilem and more), Dioxus</a>, eGUI</a>, Pax</a>, wgpu</a>, Slint</a>, and more. If you're in the Rust App Dev or UI space and would like to join future meetups, consider getting in touch!</p> Our Roadmaps for 2025</h2> Check out our next blog post</a> for roadmaps for both Project Robius and Robrix in 2025 (and beyond).</p> Acknowledgments</h2> If you made it this far, thanks for reading! You must be a true fan of Rust app dev 😊!</p> Before we depart, I'd like to thank the following key people who have been instrumental to the success of Project Robius over the past year.</p> The Makepad team: Rik Arends</a>, Eddy Bruël</a>, Sebastian Michailidis</a></li> Klim Tsoutsman</a></li> WyeWorks</a> developers: Jorge Bejar</a>, Julián Montes de Oca</a>, Facundo Mendizábal</a></li> Alex Zhang (ZhangHanDong)</a> and his team members: @alanpoon</a>, @aaravlu</a>, @tyreseluo</a>, @Guocork</a></li> Cassaundra</a></li> My colleagues who provide invaluable guidance, technical advice, and community connections: Yue Chen, Edward Tan, Sid Askary, Yong He, Mats Lundgren</li> Linebender teammembers, for technical recommendations and serving as a sounding board for exchanging ideas</li> @smarizvi110</a> and other miscellaneous contributors from the open-source community</li> </ul>

Building a ChatGPT client using Rust with Makepad

2024-06-12T00:00:00+00:00

The Rust ecosystem is making significant strides in developing first-class tools for building production-ready applications compatible with major platforms. While there's still work to be done to match the developer experience offered by other tech stacks, we can already explore our options. You'll be impressed by what Rust can accomplish today.</p>

This is the first post in a series that explores how to create cross-platform applications with Makepad</a>, a notable application framework in the Rust community. It is also one of the key projects under the Robuis initiative</a>. Although it's still evolving and some aspects are being fine-tuned for general production readiness, we can now create impressive applications in just a few steps.</p>

Create a new Makepad application</h2>

Let’s dive into it! First, create a new binary project using cargo</code>:</p>

cargo</span> new mychat
</span>cd</span> mychat/
</span></code></pre>
Now, we want to add makepad</code> as a dependency.</p>
cargo</span> add makepad-widgets</span> --git</span> https://github.com/makepad/makepad</span> --branch</span> rik
</span></code></pre>
We are now ready to create the necessary elements for an empty application to run. This requires modifying the existing main.rs and adding a few lines.</p>
The src/main.rs</code> file should contain the following:</p>
fn </span>main</span>() {
</span>    mychat::app::app_main()
</span>}
</span></code></pre>
And the src/lib.rs</code> file:</p>
pub mod </span>app;
</span></code></pre>
The src/apps.rs</code> file is the actual entrypoint for Makepad applications. Here is our first version for this file:</p>
1</td> use </span>makepad_widgets::*;
</span></td></tr>
2</td> 
</span></td></tr>
3</td> live_design! {
</span></td></tr>
4</td>     import makepad_widgets::base::*;
</span></td></tr>
5</td>     import makepad_widgets::theme_desktop_dark::*;
</span></td></tr>
6</td> 
</span></td></tr>
7</td>     App = {{App}} {
</span></td></tr>
8</td>         ui: <Window> {
</span></td></tr>
9</td>             window: {inner_size: </span>vec2</span>(</span>800</span>, </span>600</span>)},
</span></td></tr>
10</td>             pass: {clear_color: #</span>000</span>}
</span></td></tr>
11</td>         }
</span></td></tr>
12</td>     }
</span></td></tr>
13</td> }
</span></td></tr>
14</td> 
</span></td></tr>
15</td> app_main!(App);
</span></td></tr>
16</td> 
</span></td></tr>
17</td> #[</span>derive</span>(Live, LiveHook)]
</span></td></tr>
18</td> pub struct </span>App {
</span></td></tr>
19</td>     #[</span>live</span>]
</span></td></tr>
20</td>     </span>ui</span>: WidgetRef,
</span></td></tr>
21</td> }
</span></td></tr>
22</td> 
</span></td></tr>
23</td> impl </span>LiveRegister </span>for </span>App {
</span></td></tr>
24</td>     </span>fn </span>live_register</span>(</span>cx</span>: &</span>mut</span> Cx) {
</span></td></tr>
25</td>         makepad_widgets::live_design(cx);
</span></td></tr>
26</td>     }
</span></td></tr>
27</td> }
</span></td></tr>
28</td> 
</span></td></tr>
29</td> impl </span>AppMain </span>for </span>App {
</span></td></tr>
30</td>     </span>fn </span>handle_event</span>(&</span>mut </span>self</span>, </span>cx</span>: &</span>mut</span> Cx, </span>event</span>: &Event) {
</span></td></tr>
31</td>         </span>let</span> scope = &</span>mut </span>Scope::empty();
</span></td></tr>
32</td>         </span>self</span>.ui.</span>handle_event</span>(cx, event, scope);
</span></td></tr>
33</td>     }
</span></td></tr>
34</td> }
</span></td></tr></tbody></table></code></pre>
You can test our new application by executing the cargo run command. For now, you should see an empty window.</p>
Understanding our application's anatomy</h2>
Looking at the code in src/app.rs</code>, we can see various sections. One section includes a call to the live_design!</code> macro, provided by Makepad. This is where we define the UI components and layout of our application.</p>
Defining a top-level block named App</code> is essential. The behavior of this App</code> element, which represents the entire application, is determined by the Rust struct App</code>. We'll delve into this shortly. Note that our application has only one Window</code> widget instance in the App</code> definition, representing the "empty window" you see when running the application.</p>
So, how would we go about displaying a "Hello world!" message? It's simply a matter of adding a Label</code> widget instance inside the Window</code> block.</p>
3</td> live_design! {
</span></td></tr>
6</td> 
</span></td></tr>
7</td>     App = {{App}} {
</span></td></tr>
8</td>         ui: <Window> {
</span></td></tr>
9</td>             window: {inner_size: </span>vec2</span>(</span>800</span>, </span>600</span>)},
</span></td></tr>
10</td>             pass: {clear_color: #</span>000</span>}
</span></td></tr>
11</td> 
</span></td></tr>
12</td>             </span>// Adding a label displaying some text
</span></td></tr>
13</td>             body = {
</span></td></tr>
14</td>                 <Label> {
</span></td></tr>
15</td>                     margin: </span>40</span>,
</span></td></tr>
16</td>                     text: "</span>Hello World!</span>"
</span></td></tr>
17</td>                     draw_text: {
</span></td></tr>
18</td>                         color: #</span>000</span>,
</span></td></tr>
19</td>                     }
</span></td></tr>
20</td>                 }
</span></td></tr>
21</td>             }
</span></td></tr>
22</td>         }
</span></td></tr>
23</td>     }
</span></td></tr>
24</td> }
</span></td></tr></tbody></table></code></pre>
If you modified the application while it was running, you may have noticed that the changes were immediately reflected. This is thanks to Makepad's built-in Live Design feature which automatically detects UI related changes and "hot reloads" the GUI without any recompilation.</p>
All UI elements should be defined within a block named body</code>, which is specified in the Window</code> widget.</p>
Let's take a deeper look at the Rust code section of the src/app.rs</code> file:</p>
56</td> app_main!(App);
</span></td></tr>
57</td> 
</span></td></tr>
58</td> #[</span>derive</span>(Live, LiveHook)]
</span></td></tr>
59</td> pub struct </span>App {
</span></td></tr>
60</td>     #[</span>live</span>]
</span></td></tr>
61</td>     </span>ui</span>: WidgetRef,
</span></td></tr>
62</td> }
</span></td></tr>
63</td> 
</span></td></tr>
64</td> impl </span>LiveRegister </span>for </span>App {
</span></td></tr>
65</td>     </span>fn </span>live_register</span>(</span>cx</span>: &</span>mut</span> Cx) {
</span></td></tr>
66</td>         makepad_widgets::live_design(cx);
</span></td></tr>
67</td>     }
</span></td></tr>
68</td> }
</span></td></tr>
69</td> 
</span></td></tr>
70</td> impl </span>AppMain </span>for </span>App {
</span></td></tr>
71</td>     </span>fn </span>handle_event</span>(&</span>mut </span>self</span>, </span>cx</span>: &</span>mut</span> Cx, </span>event</span>: &Event) {
</span></td></tr>
72</td>         </span>let</span> scope = &</span>mut </span>Scope::empty();
</span></td></tr>
73</td>         </span>self</span>.ui.</span>handle_event</span>(cx, event, scope);
</span></td></tr>
74</td>     }
</span></td></tr>
75</td> }
</span></td></tr></tbody></table></code></pre>
The Rust code in this file is connected to the application's live UI code via the app_main</code> macro (line 56). Makepad then recognizes the App</code> struct as the one representing your application, and everything is tied together.</p>
The LiveRegister</code> trait must be implemented (lines 64-68) because the framework needs to know the locations of other live_design</code> blocks to load them. Currently, we're only including the live_design</code> block included in makepad-widgets</code>, giving us access to framework-provided widgets like Window</code> and Label</code>. However, as your project expands, you'll likely define other parts of your application in different files.</p>
Finally, the AppMain</code> trait must also be implemented, at least minimally, as we've done (see lines 70-75). For now, we've implemented the handle_event</code> function, which describes what happens when the user interacts with the application. The line self.ui.handle_event(cx, event, scope);</code> is important as it invokes the handle_event</code> handler function in the internal widget instances, such as our label instance.</p>
In makepad, if you implement a handle_event()</code> function, you effectively take control over event handling and propagation. Thus, you must explicitly pass events down into each subwidget (or "child" widget) within a widget, if you want each subwidget to be aware of the event and have the ability to handle or respond to it.
This gives you ultimate power over how events propagate throughout different UI widgets/components in the application.</p>

A crucial aspect to note in Makepad is the existence of a Draw event. This event is triggered when elements on the screen need rendering. If we fail to pass all events to child elements, they will not display because this event will not reach them.</p>
</blockquote>
Building the user interface for our chat</h2>
We aim to create a basic version of a ChatGPT client, so let's begin by designing a simple interface. It should include a text input field for the user's prompt, a submit button to send the input to the ChatGPT API, and a list of messages to display the conversation.</p>
For simplicity, we'll implement these features directly in our existing app.rs</code></a> file. In reality, larger applications would distribute different parts across multiple files. We'll cover how to organize larger applications effectively in future posts.</p>
Let's start by defining a general layout and adding the text input field and submit button.</p>
2</td> live_design! </span>{
</span></td></tr>
5</td> 
</span></td></tr>
6</td>     App = {{App}} {
</span></td></tr>
7</td>         ui: <Window> {
</span></td></tr>
10</td> 
</span></td></tr>
11</td>             body = {
</span></td></tr>
12</td>                 height: Fill,
</span></td></tr>
13</td>                 width: Fill,
</span></td></tr>
14</td>                 margin: {top: 40, bottom: 40, left: 100, right: 100},
</span></td></tr>
15</td> 
</span></td></tr>
16</td>                 show_bg: true,
</span></td></tr>
17</td>                 draw_bg: {
</span></td></tr>
18</td>                     color: </span>#330
</span></td></tr>
19</td>                 }
</span></td></tr>
20</td> 
</span></td></tr>
21</td>                 flow: Down,
</span></td></tr>
22</td>                 spacing: 20,
</span></td></tr>
23</td> 
</span></td></tr>
24</td>                 messages = <View> {
</span></td></tr>
25</td>                     height: Fill,
</span></td></tr>
26</td>                     width: Fill,
</span></td></tr>
27</td>                     margin: 20,
</span></td></tr>
28</td>                 }
</span></td></tr>
29</td> 
</span></td></tr>
30</td>                 prompt = <View> {
</span></td></tr>
31</td>                     height: Fit,
</span></td></tr>
32</td>                     width: Fill,
</span></td></tr>
33</td>                     margin: 20,
</span></td></tr>
34</td>                     spacing: 10,
</span></td></tr>
35</td> 
</span></td></tr>
36</td>                     prompt_input = <TextInput> {
</span></td></tr>
37</td>                         height: Fit,
</span></td></tr>
38</td>                         width: Fill,
</span></td></tr>
39</td>                         padding: 10,
</span></td></tr>
40</td>                         empty_message: "</span>Type a message...</span>",
</span></td></tr>
41</td>                     }
</span></td></tr>
42</td> 
</span></td></tr>
43</td>                     send_button = <Button> {
</span></td></tr>
44</td>                         height: Fit,
</span></td></tr>
45</td>                         width: Fit,
</span></td></tr>
46</td>                         padding: 10,
</span></td></tr>
47</td>                         text: "</span>Send</span>",
</span></td></tr>
48</td>                     }
</span></td></tr>
49</td>                 }
</span></td></tr>
50</td>             }
</span></td></tr>
51</td>         }
</span></td></tr>
52</td>     }
</span></td></tr>
53</td> }
</span></td></tr></tbody></table></code></pre>
We have replaced all our body block. If you run the application you should see the following:</p>
</p>
We arranged our layout using one of the Makepad's fundamental building blocks: the View</code> widget. One view can have a list of children elements to render in our interface. In our case, we added two nested views and configured the parent to have them vertically organized (indicated by flow: Down</code> in line 21).</p>
Those two children views are identified as messages</code> (line 24) and prompt</code> (line 30). Though we're not using those identifiers yet, they will be necessary for reference in the Rust code later. Observe that the first one uses height: Fill</code> and the second height: Fit</code>. This succinctly conveys that the messages section should take up all available vertical space, with each message taking only the minimum amount of vertical space required to fit the message content in., excluding the area required for the prompt</code> view. The prompt</code> view's size relies solely on its inner content.</p>

A common source of issues when working with Makepad is when you have a view sized with Fit</code>, but the inner content uses Fill</code>. This can cause something to not be displayed at all. When a widget uses Fill</code>, it needs to know the parent's size beforehand to calculate its own size. Conversely, when a widget is sized with Fit</code>, it needs to calculate the space of its content, which must be calculated without knowing the parent's size.</p>
</blockquote>
Here's an alternate way to organize our live_design</code> code:</p>
2</td> live_design! {
</span></td></tr>
5</td> 
</span></td></tr>
6</td>     Messages = <View> {
</span></td></tr>
7</td>         </span>// Empty for now
</span></td></tr>
8</td>     }
</span></td></tr>
9</td> 
</span></td></tr>
10</td>     Prompt = <View> {
</span></td></tr>
11</td>         spacing: </span>10</span>,
</span></td></tr>
12</td> 
</span></td></tr>
13</td>         <TextInput> {
</span></td></tr>
14</td>             height: Fit,
</span></td></tr>
15</td>             width: Fill,
</span></td></tr>
16</td>             padding: </span>10</span>,
</span></td></tr>
17</td>             empty_message: "</span>Type a message...</span>",
</span></td></tr>
18</td>         }
</span></td></tr>
19</td> 
</span></td></tr>
20</td>         <Button> {
</span></td></tr>
21</td>             height: Fit,
</span></td></tr>
22</td>             width: Fit,
</span></td></tr>
23</td>             padding: </span>10</span>,
</span></td></tr>
24</td>             text: "</span>Send</span>",
</span></td></tr>
25</td>         }
</span></td></tr>
26</td>     }
</span></td></tr>
27</td> 
</span></td></tr>
28</td>     App = {{App}} {
</span></td></tr>
29</td>         ui: <Window> {
</span></td></tr>
32</td> 
</span></td></tr>
33</td>             body = {
</span></td></tr>
45</td> 
</span></td></tr>
46</td>                 messages = <Messages> {
</span></td></tr>
47</td>                     height: Fill,
</span></td></tr>
48</td>                     width: Fill,
</span></td></tr>
49</td>                     margin: </span>20</span>,
</span></td></tr>
50</td>                 }
</span></td></tr>
51</td>                 prompt = <Prompt> {
</span></td></tr>
52</td>                     height: Fit,
</span></td></tr>
53</td>                     width: Fill,
</span></td></tr>
54</td>                     margin: </span>20</span>,
</span></td></tr>
55</td>                 }
</span></td></tr>
56</td>             }
</span></td></tr>
57</td>         }
</span></td></tr>
58</td>     }
</span></td></tr>
59</td> }
</span></td></tr></tbody></table></code></pre>
This approach makes our App</code> easier to read at first glance. These are specific parts of our user interface that have been assigned the aliases Messages</code> (lines 6-8) and Prompt</code>(10-26) and can be referred to from other points in the DSL. See into the body</code> definitions and check how Messages</code> (line 46) and Prompt</code> (line 51) are instantiated.</p>
We've opted to extract almost everything into these named UI blocks. However, it's worth noting that we still define (or override) the height</code>, width</code>, and margin</code> where we use Messages</code> and Prompt</code> (see lines 46-48 and 52-54). This is because these values form part of the layout rules we establish in conjunction with the parent view. But, Makepad is highly flexible, allowing you to override as much content as desired to accommodate your needs in various ways.</p>
Implementing the interaction</h2>
Currently, our application is limited to displaying user interface elements. Interaction is restricted to the text input, with no response when you click the Send</code> button. Let's enhance this interface.</p>
We need to make some changes to our UI code. We'll add some labels to the Message</code> widget to display responses. We'll also add element identifiers, allowing us to reference them from Rust code later. Check the highlighted lines to spot the elements with identifiers.</p>
2</td> live_design! {
</span></td></tr>
5</td> 
</span></td></tr>
6</td>     Messages = <View> {
</span></td></tr>
7</td>         height: Fill,
</span></td></tr>
8</td>         width: Fill,
</span></td></tr>
9</td>         padding: </span>20</span>,
</span></td></tr>
10</td> 
</span></td></tr>
11</td>         flow: Down,
</span></td></tr>
12</td>         spacing: </span>20</span>,
</span></td></tr>
13</td> 
</span></td></tr>
14</mark></td>         user_message_bubble = <RoundedView> {
</span></mark></td></tr>
15</td>             visible: </span>false</span>,
</span></td></tr>
16</td> 
</span></td></tr>
17</td>             height: Fit,
</span></td></tr>
18</td>             width: Fill,
</span></td></tr>
19</td>             padding: </span>10</span>,
</span></td></tr>
20</td>             draw_bg: {
</span></td></tr>
21</td>                 color: #</span>222
</span></td></tr>
22</td>             }
</span></td></tr>
23</td>             user_message = <Label> {
</span></td></tr>
24</td>                 height: Fit,
</span></td></tr>
25</td>                 width: Fill,
</span></td></tr>
26</td>             }
</span></td></tr>
27</td>         }
</span></td></tr>
28</td> 
</span></td></tr>
29</mark></td>         model_message_bubble = <RoundedView> {
</span></mark></td></tr>
30</td>             visible: </span>false</span>,
</span></td></tr>
31</td> 
</span></td></tr>
32</td>             height: Fit,
</span></td></tr>
33</td>             width: Fill,
</span></td></tr>
34</td>             padding: </span>10</span>,
</span></td></tr>
35</td>             draw_bg: {
</span></td></tr>
36</td>                 color: #</span>222
</span></td></tr>
37</td>             }
</span></td></tr>
38</td>             model_message = <Label> {
</span></td></tr>
39</td>                 height: Fit,
</span></td></tr>
40</td>                 width: Fill,
</span></td></tr>
41</td>             }
</span></td></tr>
42</td>         }
</span></td></tr>
43</td>     }
</span></td></tr>
44</td> 
</span></td></tr>
45</td>     Prompt = <View> {
</span></td></tr>
46</td>         height: Fit,
</span></td></tr>
47</td>         width: Fill,
</span></td></tr>
48</td>         margin: </span>20</span>,
</span></td></tr>
49</td>         spacing: </span>10</span>,
</span></td></tr>
50</td> 
</span></td></tr>
51</mark></td>         message_input = <TextInput> {
</span></mark></td></tr>
52</td>             height: Fit,
</span></td></tr>
53</td>             width: Fill,
</span></td></tr>
54</td>             padding: </span>10</span>,
</span></td></tr>
55</td>             empty_message: "</span>Type a message...</span>",
</span></td></tr>
56</td>         }
</span></td></tr>
57</td> 
</span></td></tr>
58</mark></td>         send_button = <Button> {
</span></mark></td></tr>
59</td>             height: Fit,
</span></td></tr>
60</td>             width: Fit,
</span></td></tr>
61</td>             padding: </span>10</span>,
</span></td></tr>
62</td>             text: "</span>Send</span>",
</span></td></tr>
63</td>         }
</span></td></tr>
64</td>     }
</span></td></tr>
96</td> 
</span></td></tr>
98</td> }
</span></td></tr></tbody></table></code></pre>
Note we are using visible: false</code> in some views to hide the messages bubbles (lines 15 and 30). We plan to toggle the visibility once we have some messages to display.</p>
Now we can add the Rust logic to implement the “send button”.</p>
114</td> impl </span>AppMain </span>for </span>App {
</span></td></tr>
115</td>     </span>fn </span>handle_event</span>(&</span>mut </span>self</span>, </span>cx</span>: &</span>mut</span> Cx, </span>event</span>: &Event) {
</span></td></tr>
116</td>         </span>// Added this line
</span></td></tr>
117</td>         </span>self</span>.</span>match_event</span>(cx, event);
</span></td></tr>
118</td> 
</span></td></tr>
119</td>         </span>let</span> scope = &</span>mut </span>Scope::empty();
</span></td></tr>
120</td>         </span>self</span>.ui.</span>handle_event</span>(cx, event, scope);
</span></td></tr>
121</td>     }
</span></td></tr>
122</td> }
</span></td></tr>
123</td> 
</span></td></tr>
124</td> impl </span>MatchEvent </span>for </span>App {
</span></td></tr>
125</td>     </span>fn </span>handle_actions</span>(&</span>mut </span>self</span>, </span>cx</span>: &</span>mut</span> Cx, </span>actions</span>:&Actions){
</span></td></tr>
126</td>         </span>if </span>self</span>.ui.</span>button</span>(id!(send_button)).</span>clicked</span>(&actions) {
</span></td></tr>
127</td>             </span>// Capture the text input value
</span></td></tr>
128</td>             </span>let</span> user_prompt = </span>self</span>.ui.</span>text_input</span>(id!(message_input)).</span>text</span>();
</span></td></tr>
129</td> 
</span></td></tr>
130</td>             </span>// Set the text of the user message label
</span></td></tr>
131</td>             </span>self</span>.ui.</span>label</span>(id!(user_message)).</span>set_text</span>(&user_prompt);
</span></td></tr>
132</td>             </span>self</span>.ui.</span>view</span>(id!(user_message_bubble)).</span>set_visible</span>(</span>true</span>);
</span></td></tr>
133</td> 
</span></td></tr>
134</td>             </span>// Simulate a model response
</span></td></tr>
135</td>             </span>let</span> model_response = "</span>Hello, I am a model response!</span>";
</span></td></tr>
136</td>             </span>self</span>.ui.</span>label</span>(id!(model_message)).</span>set_text</span>(model_response);
</span></td></tr>
137</td>             </span>self</span>.ui.</span>view</span>(id!(model_message_bubble)).</span>set_visible</span>(</span>true</span>);
</span></td></tr>
138</td> 
</span></td></tr>
139</td>             </span>self</span>.ui.</span>redraw</span>(cx);
</span></td></tr>
140</td>         }
</span></td></tr>
141</td>     }
</span></td></tr>
142</td> }
</span></td></tr></tbody></table></code></pre>
Let's examine each of these code blocks in detail. First, we added a call to self.match_event(cx, event)</code> in line 117, which allows us to use the simpler form of Event matching/handling. This requires us to implement the MatchEvent</code> trait for the App</code> struct, which we can then use to check for and handle events like a Button</code> click.</p>
Next, we implement handle_actions</code> to handle the clicked action</em> emitted by the button when the user clicks it (see lines 125-141). The key idea in Makepad is that widgets consume events</em> from other sources and then can emit actions</em> as needed to communicate with other widgets. In this case, the button instance already received and handled a click event and has emitted a related clicked action</em>.
We rely on the clicked(&actions)</code> function (line 126) to check if the received actions were actually emited by this button instance.</p>
You may notice how we are relying on the identifiers we have in our live_design</code> counterpart. Things like self.ui.text_input(id!(message_input))</code>, self.ui.label(id!(user_message))</code> and self.ui.view(id!(model_message_bubble))</code>. This is the Makepad query system for UI elements from Rust code, which is very confortable to use. Just remember you need to use the appropriate function depending on the widget type you’re looking for. In other words, self.ui.view(id!(message_input))</code> won’t return anything because the message_input</code> id was used for a TextInput</code> rather than a View</code>.</p>
As a final note, Makepad has a drawing mode which is quite explicit. So, it is our call to indicate to the framework that there were changes in the labels and views instances that needs to be redraw. Hence, we have an invocation to do it: self.ui.redraw(cx)</code>, in the line 139. This is a very simple way to “redraw everything” that is not the most efficient way if you have a much more elaborated UI where only a tiny portion has changed, but it is probably a good way to start for now. Nothing stops you to try later to invoke redraw</code> in the individual instances of Label</code>, View</code> and TextInput</code> as necessary.</p>
Since we have changed Rust code we are required to recompile our application and run it again to see the changes. Hopefully, you will notice how fast Makepad applications recompile! This is a luxury to have in the Rust ecosystem thanks to the amount of care the Makepad team puts on it.</p>
</p>
It is working! We need to have a model delivering smarter responses now 🙂</p>
ChatGPT interaction</h2>
It’s time to bring real conversation content to our app. We’re going to use the ChatGPT public API, so you will need to generate a key by signing into platform.openai.com</a>. Note the number of allowed requests is based on your current usage tier</a>. Using the free tier, you may have to wait a bit while testing because you only get 3 requests per minute. In any case, our implementation will catch error responses and display them so the user always knows what's going on.</p>
Once you have your API key, let’s implement the request to obtain a model response, by following the official documentation</a>.</p>
1</td> use </span>makepad_widgets::*;
</span></td></tr>
2</mark></td> use </span>makepad_micro_serde::*;
</span></mark></td></tr>
3</td> 
</span></td></tr>
125</td> impl </span>MatchEvent </span>for </span>App {
</span></td></tr>
126</td>     </span>fn </span>handle_actions</span>(&</span>mut </span>self</span>, </span>cx</span>: &</span>mut</span> Cx, </span>actions</span>:&Actions){
</span></td></tr>
127</td>         </span>if </span>self</span>.ui.</span>button</span>(id!(send_button)).</span>clicked</span>(&actions) {
</span></td></tr>
128</td>             </span>// Capture the text input value
</span></td></tr>
129</td>             </span>let</span> user_prompt = </span>self</span>.ui.</span>text_input</span>(id!(message_input)).</span>text</span>();
</span></td></tr>
130</td> 
</span></td></tr>
131</td>             </span>// Set the text of the user message label
</span></td></tr>
132</td>             </span>self</span>.ui.</span>label</span>(id!(user_message)).</span>set_text</span>(&user_prompt);
</span></td></tr>
133</td>             </span>self</span>.ui.</span>view</span>(id!(user_message_bubble)).</span>set_visible</span>(</span>true</span>);
</span></td></tr>
134</td>             </span>self</span>.ui.</span>redraw</span>(cx);
</span></td></tr>
135</td> 
</span></td></tr>
136</td>             </span>// Replacing the hardcoded response with a real one
</span></td></tr>
137</td>             </span>send_message_to_chat_gpt</span>(cx, user_prompt);
</span></td></tr>
138</td>         }
</span></td></tr>
139</td>     }
</span></td></tr>
140</td> }
</span></td></tr>
141</td> 
</span></td></tr>
142</td> fn </span>send_message_to_chat_gpt</span>(</span>cx</span>: &</span>mut</span> Cx, </span>message</span>: String) {
</span></td></tr>
143</td>     </span>let</span> completion_url = "</span>https://api.openai.com/v1/chat/completions</span>".</span>to_string</span>();
</span></td></tr>
144</td>     </span>let</span> request_id = live_id!(SendChatMessage);
</span></td></tr>
145</td>     </span>let mut</span> request = HttpRequest::new(completion_url, HttpMethod::</span>POST</span>);
</span></td></tr>
146</td> 
</span></td></tr>
147</td>     request.</span>set_header</span>(
</span></td></tr>
148</td>         "</span>Content-Type</span>".</span>to_string</span>(),
</span></td></tr>
149</td>         "</span>application/json</span>".</span>to_string</span>()
</span></td></tr>
150</td>     );
</span></td></tr>
151</td> 
</span></td></tr>
152</td>     request.</span>set_header</span>(
</span></td></tr>
153</td>         "</span>Authorization</span>".</span>to_string</span>(),
</span></td></tr>
154</mark></td>         "</span>Bearer <YOUR_ACCESS_KEY></span>".</span>to_string</span>()
</span></mark></td></tr>
155</td>     );
</span></td></tr>
156</td> 
</span></td></tr>
157</td>     request.</span>set_json_body</span>(ChatPrompt {
</span></td></tr>
158</td>         messages: vec![Message {content: message, role: "</span>user</span>".</span>to_string</span>()}],
</span></td></tr>
159</td>         model: "</span>gpt-3.5-turbo</span>".</span>to_string</span>(),
</span></td></tr>
160</td>         max_tokens: </span>100
</span></td></tr>
161</td>     });
</span></td></tr>
162</td> 
</span></td></tr>
163</td>     cx.</span>http_request</span>(request_id, request);
</span></td></tr>
164</td> }
</span></td></tr>
165</td> 
</span></td></tr>
166</td> #[</span>derive</span>(SerJson, DeJson)]
</span></td></tr>
167</td> struct </span>ChatPrompt {
</span></td></tr>
168</td>     </span>pub </span>messages</span>: Vec<Message>,
</span></td></tr>
169</td>     </span>pub </span>model</span>: String,
</span></td></tr>
170</td>     </span>pub </span>max_tokens</span>: </span>i32
</span></td></tr>
171</td> }
</span></td></tr>
172</td> 
</span></td></tr>
173</td> #[</span>derive</span>(SerJson, DeJson)]
</span></td></tr>
174</td> struct </span>Message {
</span></td></tr>
175</td>     </span>pub </span>content</span>: String,
</span></td></tr>
176</td>     </span>pub </span>role</span>: String
</span></td></tr>
177</td> }
</span></td></tr>
178</td> 
</span></td></tr></tbody></table></code></pre>

Remember to use your own OpenAI access key in the line 154</p>
</blockquote>
With the addition of send_message_to_chat_gpt()</code> (lines 142-164), we can send a request to the ChatGPT API server. Note that we’re not yet handling the response so we can focus on the request part. The cx.http_request</code> (line 163) is the mechanism in Makepad to issue regular HTTP requests in a non-blocking manner. This ensures that our application UI won’t be blocked while the response is still pending.</p>
The HttpRequest::set_json_body()</code> function (line 157) receives a struct representing the JSON format expected by the server. Note that we define the ChatPrompt</code> struct (line 167) for this purpose, and then derive SerJson</code> and DeJson</code> on them to automatically generate efficient JSON parsing logic for them. You can think of those traits as a simpler version of Serde</code>, which we are importing in the line 2.</p>
Let’s now receive the responses and update the user interface:</p>
114</td> impl </span>MatchEvent </span>for </span>App {
</span></td></tr>
129</td> 
</span></td></tr>
130</td>     </span>fn </span>handle_network_responses</span>(
</span></td></tr>
131</td>         &</span>mut </span>self</span>,
</span></td></tr>
132</td>         </span>cx</span>: &</span>mut</span> Cx,
</span></td></tr>
133</td>         </span>responses</span>: &NetworkResponsesEvent
</span></td></tr>
134</td>     ) {
</span></td></tr>
135</td>         </span>let</span> label = </span>self</span>.ui.</span>label</span>(id!(model_message));
</span></td></tr>
136</td>         </span>for</span> event in responses {
</span></td></tr>
137</td>             </span>match </span>&event.response {
</span></td></tr>
138</td>                 NetworkResponse::HttpResponse(response) => {
</span></td></tr>
139</td>                     </span>match</span> event.request_id {
</span></td></tr>
140</td>                         live_id!(SendChatMessage) => {
</span></td></tr>
141</td>                             </span>if</span> response.status_code == </span>200 </span>{
</span></td></tr>
142</td>                                 </span>let</span> chat_response =
</span></td></tr>
143</td>                                     response.get_json_body::<ChatResponse>().</span>unwrap</span>();
</span></td></tr>
144</td>                                 label.</span>set_text</span>(
</span></td></tr>
145</td>                                     &chat_response.choices[</span>0</span>].message.content
</span></td></tr>
146</td>                                 );
</span></td></tr>
147</td>                             } </span>else </span>{
</span></td></tr>
148</td>                                 label.</span>set_text</span>(&format!(
</span></td></tr>
149</td>                                     "</span>Failed to connect with OpenAI: </span>{:?}</span>",
</span></td></tr>
150</td>                                     response.</span>get_string_body</span>()
</span></td></tr>
151</td>                                 ));
</span></td></tr>
152</td>                             }
</span></td></tr>
153</td> 
</span></td></tr>
154</td>                             </span>self</span>.ui.</span>view</span>(id!(model_message_bubble)).</span>set_visible</span>(</span>true</span>);
</span></td></tr>
155</td>                             </span>self</span>.ui.</span>redraw</span>(cx);
</span></td></tr>
156</td>                         },
</span></td></tr>
157</td>                         _ => (),
</span></td></tr>
158</td>                     }
</span></td></tr>
159</td>                 }
</span></td></tr>
160</td>                 NetworkResponse::HttpRequestError(error) => {
</span></td></tr>
161</td>                     label.</span>set_text</span>(
</span></td></tr>
162</td>                         &format!("</span>Failed to connect with OpenAI </span>{:?}</span>", error)
</span></td></tr>
163</td>                     );
</span></td></tr>
164</td>                     </span>self</span>.ui.</span>view</span>(id!(model_message_bubble)).</span>set_visible</span>(</span>true</span>);
</span></td></tr>
165</td>                     </span>self</span>.ui.</span>redraw</span>(cx);
</span></td></tr>
166</td>                 }
</span></td></tr>
167</td>                 _ => ()
</span></td></tr>
168</td>             }
</span></td></tr>
169</td>         }
</span></td></tr>
170</td>     }
</span></td></tr>
171</td> }
</span></td></tr>
209</td> 
</span></td></tr>
210</td> #[</span>derive</span>(SerJson, DeJson)]
</span></td></tr>
211</td> struct </span>ChatResponse {
</span></td></tr>
212</td>     </span>pub </span>id</span>: String,
</span></td></tr>
213</td>     </span>pub </span>object</span>: String,
</span></td></tr>
214</td>     </span>pub </span>created</span>: </span>i32</span>,
</span></td></tr>
215</td>     </span>pub </span>model</span>: String,
</span></td></tr>
216</td>     </span>pub </span>usage</span>: Usage,
</span></td></tr>
217</td>     </span>pub </span>choices</span>: Vec<Choice>,
</span></td></tr>
218</td>     </span>pub </span>system_fingerprint</span>: Option<String>,
</span></td></tr>
219</td> }
</span></td></tr>
220</td> 
</span></td></tr>
221</td> #[</span>derive</span>(SerJson, DeJson)]
</span></td></tr>
222</td> pub struct </span>Usage {
</span></td></tr>
223</td>     </span>prompt_tokens</span>: </span>i32</span>,
</span></td></tr>
224</td>     </span>completion_tokens</span>: </span>i32</span>,
</span></td></tr>
225</td>     </span>total_tokens</span>: </span>i32</span>,
</span></td></tr>
226</td> }
</span></td></tr>
227</td> 
</span></td></tr>
228</td> #[</span>derive</span>(SerJson, DeJson)]
</span></td></tr>
229</td> struct </span>Choice {
</span></td></tr>
230</td>     </span>message</span>: Message,
</span></td></tr>
231</td>     </span>finish_reason</span>: String,
</span></td></tr>
232</td>     </span>index</span>: </span>i32</span>,
</span></td></tr>
233</td>     </span>logprobs</span>: Option<String>,
</span></td></tr>
234</td> }
</span></td></tr></tbody></table></code></pre>
Makepad's MatchEvent</code> trait has a handle_network_responses()</code> function, and by implementing it (lines 130-171) we now have a way to network-related events. This function is quite straightforward once we define a ChatResponse</code> struct (line 211) to represent the JSON response format coming from ChatGPT.</p>
Once we retrieve the chat message from the response, we set the corresponding Label</code> instance's text (line 144). We also make sure that the parent view is visible (line 154) and everything gets redrawn (line 155). The parent visibility was hidden because we only want to display the messages bubbles once we have the messages</p>
If everything goes well with the ChatGPT API server, you should see an interaction like the following one:</p>

This is what ChatGPT knows about Makepad...</em> 🙂</p>
Wrapping up</h2>
Let's take a look at what we have achieved in a few steps. We have a basic application with an easy-to-modify look and feel. In just a few dozen lines of code, we made a working chat client that shows real responses from ChatGPT models. Now, we invite you to try different platforms beyond the native desktop platform that we demonstrated in this post. Follow the instructions in the Makepad README</a> to see how to test this application in Android, iOS, and web.</p>
In a future post, we'll cover how to implement a list of messages to make this app really feel like a true interactive chat app.</p>
Have fun hacking with Makepad and Rust!</strong></p>


Performance Benchmarking on iOS
2024-05-25T00:00:00+00:00
After testing the Performance Benchmarking</a>  of cross-platform applications written with Makepad on Android platforms, we decided to do some tests on iOS.</p>
Instead of using the exisitng mock WeChat and TaoBao apps, we had decided to focus more on graphics effects performance. For this, the team at WyeWorks had helped create a suite of graphics effects in an app called comp_demo</strong>.</p>

  
Graphics Effects</h2>
The effects below have been implemented in all three apps:</p>

Rounded Corner</li>
Bitmap Image RC</li>
Bitmap Text Scaling</li>
Vector Text Scaling</li>
Transparency effects</li>
Control Shadow</li>
Path Shadow</li>
Control Stroke</li>
</ul>
Implementations</h2>
The same set of features were implemented using Swift UI to run as iOS native application, using Flutter to run on iOS.</p>
Using Makepad, the same set of features were implemented and also ran on iOS.</p>
Devices</h2>
All 3 apps and tests were ran on the same iPhone 12 and iPhone 13 devices. The devices themselves were "clean" machines, meaning a new machine wihtout any extra applications installed, just the three test apps.</p>
The phones were running iOS 17.3.</p>
  </div>
  
    
      
    </a>
  </div>
</div>


Process</h2>
Collecting performance metrics on iOS is not as straightforward as on Android. Android's Perfetto tool automates most of the process of collecting and presenting the metrics, both in UI and JSON data formats. But on iOS, the Xcode's Instrumentation tool mainly deals with UI displays of real-time performance graphs. While this is helpful in some situations, it is not sufficient for the metrics that we wanted to collect.</p>
Hence, we had to utilize the xcrun xctrace</code> commands to record metrics we are interested in, then use export to extract the metrics from the proprietary binary file formats into an xml file, and then convert it into csv format in order to be more human readable using a spreadsheet application.</p>
After running through the process several times, we devised scripts to automate as many of these manual steps as possible. This included running one command to start the recording process to do the test. And then one script to extract the proper data, and one more at the end to summarize the data that we're interested in, such as the CPU %, Memory usage, etc.</p>
Results</h2>
The results were interesting to see. First, for most of the Swift UI version of the application, the data were not readily or easily seen. It seems that iOS "hides" or "combines" the actual application usage data with the operating system, such that it is not apparent which work was actually peformed and counted as part of the application and which were counted as OS.</p>
</a>
(In the table above, red background means good, yellow means bad)</p>
So the more comparable test results are between the Makepad and Flutter apps. As both of these applications come with their own rendering layer, and do not utilize iOS's own native rendering.</p>
Rounded Corners and Text Scaling</h3>
From these results, it can be seen that in general, Makepad app's CPU % usage is often ~50% less than what the Flutter app's were. On the other hand, Makepad app's GPU % usage is often ~30% higher.</p>
Note that memory usage includes both the CPU and GPU memory in this case. Makepad's memory usage is in general much higher than the others. Part of the reason is that currently Makepad still bundles its own big font file as part of the app, which results in about 40MB of memory usage on its own.</p>
As part of the Project Robius's platform abstraction efforts, we hope to add feature for Makepad (and other Rust UI applications) to be able to easily use the underlying platform's system fonts. This will help reduce the memory footprint by a large amount.</p>
These results are actually very similar to those seen in the previous Performance Benchmarking</a> of scrolling tests on Android. At that time, Makepad apps also exhibited the lower CPU but higher GPU usage pattern.</p>
This can be attributed to Makepad's architecture design of doing some work on CPU but doing a lot of other graphics manipulation on the GPU directly by utilizing shader programs. This allows certain graphics operations to be as simple as changing a variable, without the need for repeated drawcalls. Makepad also optimizes drawcalls by minimizing the number of them. It also has the flexibility of allowing users to write shader programs to execute on the GPU.</p>
Shadow Effects</h3>
For the shadow effects, we see the huge advantage that Makepad app has over the equivalent Flutter and even Swift apps.  Makepad app is the only one that managed a steady 60 fps rendering, while the Swift and Flutter apps struggled to stay in the ~45 fps range.</p>
The large CPU % spike also indicates that it is working extra hard to perform these graphics shadow effects.</p>
Summary</h2>
For a relatively new (~4 years) UI framework written in Rust, Makepad's performance can be seen as quite impressive, especially consdering it has been developed mainly by a team of 3 people.</p>
Part of the performance advantages can be attributed to the efficiency of the programming language Rust itself. But a major reason can be attributed to Makepad's novel design approach of having a mixed immediate mode + retained mode rendering system. Currently, the framework has not even begun to optimize specifically for performance tests. For example, the higher memory footprint is a current design decision, and has room for furter optimization according to Rik</a>.</p>
We're looking forward to continuing to work closely with the Makepad team to bring the best cross-platform performance UI framework in Rust to all developers.</p>
References</h2>
All tests were performed on the iOS build of the Makepad comp_demo app, running on iPhone 12 and iPhone 13.</p>
Makepad comp_demo app:
https://github.com/project-robius/comp_demo</code></a></p>
iOS Benchmarking scripts:
https://github.com/project-robius/benchmarking-ios</code></a></p>
Makepad Framework:
https://github.com/makepad/makepad</code></a></p>


Performance Improvements with SDF
2024-01-29T00:00:00+00:00
Recently, the Makepad</a> framework updated its font handling with SDF generation algorithms. We decided to test out the WeChat and TaoBao apps built using this latest iteration of the framework to see if there are any changes to performance. The tests are same as before: fast scrolling of the main content screen back and forth within a 10 second window.</p>
The WeChat and TaoBao apps' code have pretty much stayed the same. The only change was made last month to work with Makepad's then-new MatchEvent()</code> event handling paradigm, which simplified the code base and reduced the size of it by 20~30%.</p>
The SDF algorithm changes in Makepad Framework itself is transparent to the application developer. It is a Makepad internal performance optimization.</p>
Results Summary</h2>
Makepad TaoBao</h3>
</p>
As can be seen from the preliminary results, the CPU Cycles of the application reduced by about 12%, while the average and max CPU frequency went up by 18% and 56% respectively. This is because the SDF generation takes more CPU processing initially to process the fonts, but once it's done, the overall CPU usage is reduced during subsequent rendering.</p>
The CPU memory usage increased by an insignificant amount. But the GPU memory usage both reduced by over 23%. This is a significant improvement over the old Makepad way of using the Font Atlas feature. The dynamic SDF generation algorithm uses less than 1/4 of the GPU memory, a huge improvement.</p>
Makepad WeChat</h3>
</p>
Unlike the TaoBao app, the WeChat app is more "text-centric", with less images and mostly text-based rendering. The improvements here are more apparent.</p>
The CPU cycles reduced by 24%, while the CPU frequency increases by average of 15% and max of 21% respectively.</p>
The GPU memory usage reductions were even more significant, with 35% less average memory usage and 32% decrease in max memory usage.</p>
The detailed chart of each of the metrics follows below.</p>
Individual Graphs</h2>
CPU Cycles</h3>
On the CPU cycle, we see the significant reduction of CPU cycles with the new SDF algorithm as compared to the old Font Atlas rendering.</p>
For the mostly text-based app such as WeChat, the results are more significant.</p>

(For all graphs, the right side bar(s) is the new Makepad with SDF.)</em></p>
</blockquote>

</tbody></table>

  
    
      
    </a>
  </div>
  
    
      
    </a>
  </div>
</div>
CPU Frequency</h3>
The only metric where the new method resulted in higher values due to running of the SDF algorithm.</p>
Makepad TaoBao</th> Makepad WeChat</th></tr></thead>

</tbody></table>

  
    
      
    </a>
  </div>
  
    
      
    </a>
  </div>
</div>
CPU Memory Usage</h3>
The CPU memory usage had minimal increases compared to the previous method.</p>
Makepad TaoBao</th> Makepad WeChat</th></tr></thead>

</tbody></table>

  
    
      
    </a>
  </div>
  
    
      
    </a>
  </div>
</div>
GPU Memory Usage</h3>
The GPU memory shows reflects the biggest difference between the old and new algorithms. Both averagea and maximum GPU memory usage were reduced by over 30%.</p>
Makepad TaoBao</th> Makepad WeChat</th></tr></thead>

</tbody></table>

  
    
      
    </a>
  </div>
  
    
      
    </a>
  </div>
</div>
Conclusion</h2>
The results of these tests validates the benefits of the SDF generation algorithm that the latest Makepad has incorporated. CPU Cycles were reduced, indicating less processing after the initial ramp up. With SDF, Makepad can also cache the generation of the SDF on local storage to lower the CPU usage.</p>
The GPU memory use reduction is the most significant difference. The usage of the SDF generation algorithm reduced GPU memory usage by a large amount. In addition, it will also provide improved performance for dynamically scaled texts.</p>
References</h2>
All tests were performed on the Android build of the Makepad TaoBao and Makepad WeChat apps, running on Google Pixel 7 Pro.  The tests were run a minimum of 6 times each, and the results were averaged.</p>
Makepad WeChat sample app:
https://github.com/project-robius/makepad_wechat</code></a></p>
Makepad TaoBao sample app:
https://github.com/project-robius/makepad_taobao</code></a></p>
Android Benchmarking scripts:
https://github.com/project-robius/benchmarking-android</code></a></p>
Makepad Framework:
https://github.com/makepad/makepad</code></a></p>
SDF Generation Algorithm:
https://github.com/LykenSol/sdfer</code></a></p>


Performance Benchmarking - Part 2
2023-11-17T00:00:00+00:00
Several days ago we did a performance benchmark</a> test between sample apps written in Android "native" vs. Makepad. Afterwards, we were given access to another version of the sample TaoBao app with more functionality, including more animation/video content in the scroll list, and the ability to turn toggle some of the special effects.</p>
So we did the same scrolling tests with this new version, both with the special features turned on, as well as with all features turned OFF, in order to compare the differences.</p>
Updated Results</h2>
</p>
As can be seen from the results, they seem similar and consistent with our previous results from the image manipulation benchmarks.</p>
CPU Processing</h3>
</p>
On the CPU cycle, the sample Makepad WeChat app shows very little cycles used, having less than half of the Android "native" sample apps.</p>
The Native TaoBao apps were similar to the Official TaoBao in that they use a lot more CPU cycles. This also results in higher CPU frequency values.</p>
When special functionality are turned OFF, the Native TaoBao used ~40% less cycles. While the full-effect TaoBao used slightly more than the Official TaoBao app.</p>
</p>
The CPU frequencies also reflect that Makepad uses much less than the Android native counterparts.</p>
Not much difference were noticed with the new Native TaoBao versions for GPU memory usage.</p>
Memory Usage</h3>
The CPU memory usage is more similar among the Android native and Makepad sample applications, with Makepad apps using slightly less CPU memory. This is consistent with less usage of the CPU cycles as well.</p>
As expected, the new Native TaoBao performed noticeably better when its special effects were turned off.</p>
</p>
The GPU memory shows that Makepad apps use more memory than the Android native apps. This is currently the only area where Makepad is not as efficient. However, the Makepad team is currently working on an improvement that will decrease this usage in the near future.</p>
Not much difference were noticed with the new Native TaoBao versions for GPU memory usage.</p>
</p>
Conclusion</h2>
Being a more fully developed Android Native TaoBao app, we noticed much smoother and faster scrolling speed and effects compared to the previous simpler "native" apps. The performance characteristics of this version matches more closely to the official Play Store TaoBao, though with less CPU memory footprint.</p>
Overall, the results still reflect what we had found in our previous benchmarking tests. The Makepad versions of the apps have consistently outperformed all other versions, at least in terms of pure scrolling tests.</p>
References</h2>
Makepad WeChat sample:
https://github.com/project-robius/makepad_wechat</code></a></p>
Makepad TaoBao sample:
https://github.com/project-robius/makepad_taobao</code></a></p>


Performance Benchmarking
2023-11-13T00:00:00+00:00
The last time we did a performance benchmark test, it was a few months ago. At that time we used only one simple application, an image manipulation program. During the GOSIM Workshop in September, I had mentioned that we would be doing more performance benchmarking tests and we have done just that.</p>
This time we used Makepad to write a sample WeChat application and a sample TaoBao application. The main test criteria is to stress the "scrolling" feature of the apps. We also had access to two sample applications of the same written in Android "native".</p>
Test Methodology</h2>
Scrolling is one of the most important and common operations for mobile applications. For WeChat, a messaging application, and TaoBao, an e-commerce application, a smooth scrolling experience is often one of the key factors in user satisfaction with the application.</p>
We benchmarked similar performance metrics as last time using Google Perfetto</a> tool. These include:</p>

CPU Cycles</li>
CPU Frequency (Average & Max)</li>
CPU Memory (Average & Max)</li>
GPU Memory (Average & Max)</li>
</ul>
To test, we use the applications' main message or product list page. We exercise the scrolling by quickly swiping up and down on this scrollable page. We first quickly swipe for 6 to 10 times up (to make the list go down) and then 6 to 10 times down and repeat, until we reach the 10 seconds sample time.</p>
(Note that in real applications the scrolling list might have network dependencies such as loading of images, etc. But for the sample applications, the images are cached locally to the mobile app so there’s no network latency or variance.)</em></p>
In order to have some more data points, we also benchmarked the official TaoBao app from the Google Play Store.</p>
Results</h2>
</p>
As can be seen from the results, they seem similar and consistent with our previous results from the image manipulation benchmarks.</p>
CPU Processing</h3>
</p>
On the CPU cycle, the sample Makepad WeChat app shows very little cycles used, having less than half of the Android native sample app.</p>
</p>
The CPU frequencies also reflect that Makepad uses much less than the Android native counterparts.</p>
Memory Usage</h3>
The CPU memory usage is more similar among the Android native and Makepad sample applications, with Makepad apps using slightly less CPU memory. This is consistent with less usage of the CPU cycles as well.</p>
</p>
The GPU memory shows that Makepad apps use more memory than the Android native apps. This is currently the only area where Makepad is not as efficient. However, the Makepad team is currently working on an improvement that will decrease this usage in the near future.</p>
</p>
Conclusion</h2>
Being a fully functioning application, the official TaoBao app had a much higher CPU cycle and CPU memory than the rest.</p>
Also, there seems to be multiple processes related to the TaoBao application. We only counted the main TaoBao process. If we were to add the data from the TaoBao GPU process, then it will increase the amount of GPU memory usage by about 70 MB.</p>
Overall, the results are consistent to what we found in our early benchmarking test. As Makepad framework continues to improve and evolve, the numbers will undoubtably change. We will periodically run more of these performance benchmarking and post our updates.</p>
References</h2>
Makepad WeChat sample:
https://github.com/project-robius/makepad_wechat</code></a></p>
Makepad TaoBao sample:
https://github.com/project-robius/makepad_taobao</code></a></p>

Makepad TaoBao</th> Makepad WeChat</th></tr></thead>

Our Roadmaps for 2025</h2>
Check out our next blog post</a> for roadmaps for both Project Robius and Robrix in 2025 (and beyond).</p>

Performance Benchmarking on iOS

Implementations</h2>
The same set of features were implemented using Swift UI to run as iOS native application, using Flutter to run on iOS.</p>
Using Makepad, the same set of features were implemented and also ran on iOS.</p>

Performance Improvements with SDF

Individual Graphs</h2>

CPU Frequency</h3>
The only metric where the new method resulted in higher values due to running of the SDF algorithm.</p>

CPU Memory Usage</h3>
The CPU memory usage had minimal increases compared to the previous method.</p>

GPU Memory Usage</h3>
The GPU memory shows reflects the biggest difference between the old and new algorithms. Both averagea and maximum GPU memory usage were reduced by over 30%.</p>

Performance Benchmarking - Part 2

References</h2>
Makepad WeChat sample: `https://github.com/project-robius/makepad_wechat</code></a></p>`
`Makepad TaoBao sample:https://github.com/project-robius/makepad_taobao</code></a></p>`

Performance Benchmarking

CPU Processing</h3>
</p>
On the CPU cycle, the sample Makepad WeChat app shows very little cycles used, having less than half of the Android native sample app.</p>
</p>
The CPU frequencies also reflect that Makepad uses much less than the Android native counterparts.</p>

References</h2>
Makepad WeChat sample: `https://github.com/project-robius/makepad_wechat</code></a></p>`
`Makepad TaoBao sample:https://github.com/project-robius/makepad_taobao</code></a></p>`

Our Roadmaps for Project Robius & Robrix in 2025

Project Robius in 2024: one year of Rust App Dev

Building a ChatGPT client using Rust with Makepad

Our Roadmaps for Project Robius & Robrix in 2025

Acknowledgments</h2> I'd like to thank the following key people who have been instrumental to the success of Project Robius over the past year, and who will undoubtedly help it flourish in 2025.</p>

Project Robius in 2024: one year of Rust App Dev

2. Apps built in 2024 using Makepad + Robius</h2> We (with help from many collaborators) have built both small proof-of-concept demo apps and larger "flagship" apps using Makepad + Robius. Two of the most complex flagship apps we've been developing in 2024 are:</p>

Building a ChatGPT client using Rust with Makepad

Performance Benchmarking on iOS

Graphics Effects</h2> The effects below have been implemented in all three apps:</p>

Implementations</h2> The same set of features were implemented using Swift UI to run as iOS native application, using Flutter to run on iOS.</p> Using Makepad, the same set of features were implemented and also ran on iOS.</p>

Devices</h2> All 3 apps and tests were ran on the same iPhone 12 and iPhone 13 devices. The devices themselves were "clean" machines, meaning a new machine wihtout any extra applications installed, just the three test apps.</p> The phones were running iOS 17.3.</p> </div>

Summary</h2> For a relatively new (~4 years) UI framework written in Rust, Makepad's performance can be seen as quite impressive, especially consdering it has been developed mainly by a team of 3 people.</p>

Performance Improvements with SDF

Individual Graphs</h2>

CPU Frequency</h3> The only metric where the new method resulted in higher values due to running of the SDF algorithm.</p>

CPU Memory Usage</h3> The CPU memory usage had minimal increases compared to the previous method.</p>

GPU Memory Usage</h3> The GPU memory shows reflects the biggest difference between the old and new algorithms. Both averagea and maximum GPU memory usage were reduced by over 30%.</p>

References</h2> All tests were performed on the Android build of the Makepad TaoBao and Makepad WeChat apps, running on Google Pixel 7 Pro. The tests were run a minimum of 6 times each, and the results were averaged.</p>

Performance Benchmarking - Part 2

Performance Benchmarking

Test Methodology</h2> Scrolling is one of the most important and common operations for mobile applications. For WeChat, a messaging application, and TaoBao, an e-commerce application, a smooth scrolling experience is often one of the key factors in user satisfaction with the application.</p>

CPU Processing</h3> </p> On the CPU cycle, the sample Makepad WeChat app shows very little cycles used, having less than half of the Android native sample app.</p> </p> The CPU frequencies also reflect that Makepad uses much less than the Android native counterparts.</p>

Implementations</h2>
The same set of features were implemented using Swift UI to run as iOS native application, using Flutter to run on iOS.</p>
Using Makepad, the same set of features were implemented and also ran on iOS.</p>

CPU Frequency</h3>
The only metric where the new method resulted in higher values due to running of the SDF algorithm.</p>

CPU Memory Usage</h3>
The CPU memory usage had minimal increases compared to the previous method.</p>

GPU Memory Usage</h3>
The GPU memory shows reflects the biggest difference between the old and new algorithms. Both averagea and maximum GPU memory usage were reduced by over 30%.</p>

CPU Processing</h3>
</p>
On the CPU cycle, the sample Makepad WeChat app shows very little cycles used, having less than half of the Android native sample app.</p>
</p>
The CPU frequencies also reflect that Makepad uses much less than the Android native counterparts.</p>