Don't panic, impl Things A blog by Martijn Gribnau Zola 2026-03-08T00:00:00+00:00 https://gribnau.dev/atom.xml The Middle Ground 2026-03-08T00:00:00+00:00 2026-03-08T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/builders-and-craftspeople/ <p>Like many, I too have been spending much time thinking about (generative) AI and its values. In my field of work (Software Engineering), on online forums such as Hacker News and in blog posts, generative AI has become an almost polarized topic between the believers and the unbelievers. Of course there's a middle ground, yet as it often is, the outer ends are the most vocal, and usually are most certain of their position. I suspect this divide maps onto something deeper: how people perceive the value of their work. To some, code and software are just tools; means to an end. They're the <strong>makers</strong>, the pragmatists of our field. On the other end you have the people who do not just care what they're building, but also <em>how</em> it is built. They're <strong>craftspeople</strong><sup>1</sup><sup>2</sup>, perfectionists who care as much about how something is built as what it does.</p> <p>In companies and larger projects, I'm convinced you need both. Craftspeople tend to spend (too) much time on details. Some details matter, but not all. Usually, proceeding forward is necessary both for economical reasons, but also to learn more about the real requirements of the product. That's where makers come in. They are better at making difficult decisions, even with incomplete details. However, they also tend to "just make it work", even if that means using suboptimal solutions and architecture. In turn, future additions can be chaotic and more difficult, and in the longer term, time consuming; it can also mean it'll require more maintenance<sup>3</sup>. Where the balance rests depends on many factors. What do you produce? Are you responsible for maintenance? Do your customers care about what maintenance will cost them, or how much time it will take before a product must be replaced?<sup>4</sup></p> <p>When searching for a balance, it's unusual to find easy answers. However, I'm convinced that the answer is almost never at the edges of the spectrum. When in doubt, choose some kind of middle ground. This is why the polarised AI debate frustrates me. On one side, people want to use AI to build as much and as fast as possible; the initial outcomes are all that matter. On the other, people refuse to engage with AI at all and dismiss it outright. Neither extreme leaves room for the conversation that actually matters (if we assume it is here to stay): when does it help, when doesn't it, and how do we work well regardless? The answer is unlikely to be found at either extreme.</p> <p><sup>1</sup> Like almost anything, almost nobody is completely on one side or the other; everything is a spectrum. Yet, no argument can be made if you need to deal with infinite exceptions. Every model is flawed in some way.</p> <p><sup>2</sup> I also wonder if, and how, any of the color personality models relate to the stance of people with respect to their opinion of generative AI. I would hypothesize that makers (pragmatists) skew red or yellow (i.e. action oriented, results focused, impatient with details), and that craftspeople (perfectionists) skew blue or green (i.e. methodical, quality focused, slower to act).</p> <p><sup>3</sup> Software projects are rarely ever done. After new features are added, maintenance is required for security updates, to keep internal and external services compatible, and to remove overhead. In that respect, software is not different from hardware, whether it's an aircraft, boat, bike, or a server rack, a graphics card or even shoes. Everything wears down. The type of maintenance changes, but not the need.</p> <p><sup>4</sup> If you (or a competitor) can write a new version or competing product at a fraction of the cost, it affects the sustainability of software businesses. Generative AI might be exactly this kind of shift and if so, it will affect where the balance rests.</p> Notes on AI and empowerment 2026-03-01T00:00:00+00:00 2026-03-01T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/ai-and-empowerment/ <p>Today, I came across the following post <a href="https://github.com/nikomatsakis/rust-project-perspectives-on-ai/blob/f4ee1bef35c32d3f8691761054d0e15b925a6fe3/src/all-comments.md#nikomatsakis">Rust Project Perspectives on use of AI</a>. I realize there are more opinions in this post, but I will only focus on Niko's opinion as it made me think about the different sides of empowerment. <sup>NB: I do not intend to pick on Niko's opinion here; there's no ill intent behind this reply. Consider it a post-dinner thought if you will.</sup></p> <p>In particular, Niko mentions:</p> <blockquote> <p>"If I had to pick one word for how I feel about using AI, it is empowered".</p> </blockquote> <p>And goes on to state:</p> <blockquote> <p>"Suddenly it feels like I can take on just about any problem -- it's not that the AI will do the work for me. It's that the AI will help me work through it and also tackle some of the drudgery. And certainly there are some areas (github actions, HTML/CSS) that would just stop me cold before, but which now are no issue -- I can build out those things and brings things to life."</p> </blockquote> <p>And to highlight one more part (you can read the full opinion in the link above):</p> <blockquote> <p>"This is not to say AI doesn't have problems. It has huge ones, power usage chief among them, but also the way irresponsible usage is creating slop PRs (not to mention the way it's being inserted in places it doesn't belong, and the ways it will be abused by governments)."</p> </blockquote> <p>The first thing I thought was; arguably especially with all the agentic programming going on, "it's not that the AI will do the work for me" sounds very much like the AI doing <em>the</em> work, or at least, some of <em>the</em> work. Is that good or bad? Hard to judge.</p> <p>But this is a point which made me think, namely about <em>senders</em> and <em>receivers</em><sup>1</sup>. That is, <em>senders</em> are the people writing the code and making PR's. The <em>receivers</em> are the people reviewing the code. So far, I feel that AI could be empowering primarily for <em>senders</em>, but is relatively more likely to be disempowering for the <em>receivers</em> (not for all of course).</p> <p>Over the last few months I've been less active in open source because emotionally, programming no longer seemed to have much value. Being on the receiving end of AI slop made reviewing code a slog. For much code written by LLM's, it's time consuming to decide whether it is slop or genuine, because LLM's make it hard to see the difference without thorough reviews. It feels, I the reviewer am often the first to go thoroughly through the commits, while I would pose, you should always review your own code before asking for the opinion of a maintainer.</p> <p>Emotionally, PR's made (largely) by AI feel different. Open source is not just about the product, but also about people. As a maintainer, I always try to make people feel welcome; them spending the time to contribute to a foreign code base, and helping them find their way, brought me a kind of joy. With the advent of LLM's this joy has been diminishing. I no longer know if I'm helping a human, or a computer. Being the <em>receiver</em> feels tiresome; disempowering.</p> <p>For the past few years, we've also rightfully seen many discussions about the mental health of Rust (and community) maintainers. It makes me wonder what the impact of LLM's have been on the mental health of Rust maintainers, the last few years.</p> <p>As it often feels like there are two vocal camps in the online circles, one pro and one con, with little room for sincere discussion, I welcome <a href="https://nikomatsakis.github.io/rust-project-perspectives-on-ai/">the discussion</a>, and will be reading the other opinions too, for new insights.</p> <p><sup>1</sup> No, not <a href="https://doc.rust-lang.org/std/sync/mpmc/index.html">these</a> senders and receivers.</p> The Value of Things 2026-03-01T00:00:00+00:00 2026-03-01T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/the-value-of-things/ <p>I've been playing with LLM's over the past few years; probably in part was just a fear of missing out.</p> <p>I've both used AI via chats, and let LLM's program agentically. My feelings are mixed. One day I will write a more detailed opinion, but hey, with all this hype, there are enough opinions<sup>1</sup>.</p> <p>One thought keeps repeating though;</p> <p>I reminisce about this lesson from one of my favorite books as a child<sup>2</sup>:</p> <blockquote> <p>"When you can have anything you want by uttering a few words, the goal matters not, only the journey to it." - Christopher Paolini (Eldest, 2005)</p> </blockquote> <p><sup>1</sup>Tons of new opinions on AI every day, find them here: <a href="https://news.ycombinator.com">news.ycombinator.com</a></p> <p><sup>2</sup>I reread the book as an adult, when the follow-up novel Murtagh came out. As a child I loved all the side-quests. Now I felt the books could have been a much, much shorter. And ugh, I still hated the amount of fan-service in the last book of the original series. <a href="https://www.fanlore.org/wiki/The_Suck_Fairy">🧚</a></p> How I use Spotify 2026-01-18T00:00:00+00:00 2026-01-18T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/spotify/ <p>I've been a Spotify subscriber for over 11 years. Over the years I've tried several of the other music apps, but Spotify stuck. It is however not an app without its pain points. On occasion I've been using the venerable <a href="https://spicetify.app/">Spicetify</a> to fix issues, like when Spotify tried to go above and beyond in promoting podcasts.Spicetify did have its disadvantages however and one is that it broke on almost every Spotify release (the Spicetify author was very fast in fixing Spicetify, but I kept needing to update my extensions too and had no desire to keep that up in the end).</p> <h2 id="how-i-use-spotify">How I use Spotify</h2> <p>One thing I've been doing since 2014 is to create a "monthly playlist" every month. During the month I add songs I've come to like (or come back to) to this playlist. I listen to this playlist a lot during the current month. When a new month arrives, I create a new playlist, and the cycle continues. This way I can re-listen to songs while they do not become to repetitive. I tend to start disliking songs I've heard too much after that (like the ones I use for my alarm clock haha). These playlists also tend to stay manageable; a playlist has between 100 and 200 songs at the end of the month.</p> <p>Each monthly playlist is sorted in a yearly folder, e.g. "2026 • Monthly". I've recently added another level and placed the 2014 to 2019 folders in a "201X • Monthly" folder, mostly because having many top level playlists becomes a hassle on mobile.</p> <p>For some special occasions, or moods I also create playlists, which I put in a folder called "Trove". These playlists are more situational, and I tend to listen to these less.</p> <p>I have some assorted folders like "Subscribed" for playlists I didn't create myself, Spotify mixes I like, and the yearly "your top songs" list.</p> <p>Finally, I have three folders for favorites; from "kind of favorite" to "favorite" to "favorite of favorites". All very subjective, and mood dependent, I suppose.</p> <p>I always, always, use custom sort order where I can. I'm used to the consistent ordering and want nothing else.</p> <p><img src="/img/spotify_playlists.jpg" alt="Spotify playlists" /></p> <h2 id="the-good-parts">The good parts</h2> <h3 id="global-music-library">Global music library</h3> <p>Spotify is pretty great in this regard. It has an almost perfect library. Songs are rarely removed, unlike for example film centric platforms. Songs are also usually available worldwide, and not as much fenced by region (afaik), unlike say e-book or audiobook platforms.</p> <h3 id="audio-quality">Audio quality</h3> <p>Disclaimer: I'm not an audiophile. I've been streaming Spotify on "Very high" and more recently sometimes on "Lossless" too; I've can't complain about the quality. It's good.</p> <h3 id="built-in-lyrics">Built-in lyrics</h3> <p>This is a keeper. When actively listening to music (as opposed to in the background), I like to understand what is being sung. It was gone for some long years. I hope it will stay this time around. I never quite understood why a third party service is necessary for it either. The line synchronization is nice I suppose, but hardly necessary.</p> <h3 id="when-consistency-works">When consistency works</h3> <p>Despite some UI refreshes and a migration of the user interface to React, the Spotify user interface has been recognizably the same in years. Commonality and consistency is a great perk in the toolbox of the power user. Spotify may not have brought anything new to the table, with respect to the interface, but I would pose that that's a good thing. I couldn't stand the Play Music interface for example, when I tried it a few years ago.</p> <p>The base layout has everything you need. A list of playlists on the left. A list of (sometimes grouped) songs on the right. It's tried and true.</p> <h2 id="pain-points">Pain points</h2> <h3 id="podcasts">Podcasts</h3> <p>This one is fairly common. It was the main reason to turn to Spicetify originally.</p> <p>I listen to podcasts almost exclusively on my phone, and for better or worse have been using the Pocket casts app for that. I'm used to it, and do not intend to use Spotify anytime soon. Spicetify allowed me to filter podcasts out of the desktop app. But as I wrote earlier I stopped using it. It wasn't a perfect solution anyhow, as it only covered my desktop use.</p> <p>I feel Spotify has been pushing podcasts less than they used to a few years back, and I'm glad. I considered unsubscribing several times because of how much podcasts were promoted, and interfered with my flow.</p> <p>In "My Library", there still is this mandatory "Your Episodes" playlist (can't be much of "my" library if I don't have the power to remove it). I wish I could remove it natively.</p> <p><img src="/img/spotify_your_episodes.jpg" alt="Spotify &quot;your episodes&quot;" /></p> <p><img src="/img/spotify_your_episodes_pin.jpg" alt="Spotify &quot;your episodes&quot; can only be pinned and unpinned" /></p> <p><img src="/img/spotify_your_episodes_cant_move.jpg" alt="Spotify &quot;your episodes&quot; can not be moved" /></p> <p>On my desktop home page, I luckily only get music recommendations. I don't even need to select the "Music" filter for that. It feels like a bug, but to me it's a feature. On mobile however I explicitly need to filter music. As a result, I don't tend to use "home" on mobile much. As a result, I mostly discover music either via the "Discover weekly" playlist or the "Go to song radio" feature.</p> <h3 id="add-song-to-playlist-on-mobile">"Add song to playlist" on mobile</h3> <p>This feature of "adding a song to a playlist" is something that I keep disliking (on mobile anyways). It's a place where I feel the app authors keep telling me they know better than I do, to which playlist I would want to add a song.</p> <p>The thing is, they do not permit me to browse in my custom sort order. Instead, they offer "Recently updated", "Recently added", "Alphabetical" and "Recently played". The only consistent ordering is "Alphabetic", which is approximately the reverse of my custom structure (older years for monthly playlists come first - while I want to add it to a more recent one). Arguably "Recently updated" works well for my workflow, but I value consistent ordering even more.</p> <p><img src="/img/spotify_add_to_playlist_sort.jpg" alt="add song to playlist sorting options" /></p> <p>In my library I can thankfully use a custom order:</p> <p><img src="/img/spotify_library_sort.jpg" alt="library sorting options" /></p> <p>In addition to the sorting order, the "add to playlist" dialogs also have this odd feature where each folder with playlist in it, is virtualized (like a symlink) in the top level. As a result, you can navigate to playlists in deeper folders via their parents, or directly. But for someone who has a ton of playlists, and someone who added all these folders to create order, this feature just adds chaos.</p> <p>For example, this is what I see from my library:</p> <p><img src="/img/spotify_add_to_playlist_library.jpg" alt="Playlists in my library" /></p> <p>But "add to library" shows these deeper level folders at the top-level too:</p> <p><img src="/img/spotify_add_to_playlist_virt.jpg" alt="Folders virtualized to the top level" /></p> <p><a href="/video/spotify_add_to_playlist_virtual_folders.mp4">This</a> video shows both (watch the folder titles).</p> <h3 id="when-keyboard-shortcuts-don-t-match-the-highlighted-action">When keyboard shortcuts don't match the highlighted action</h3> <p>With my playlist heavy workflow, I use the "add song to playlist" flow a lot. Spotify warns you when you add a duplicate (but doesn't block you). This is a great feature.</p> <p><img src="/img/spotify_add_double.jpg" alt="Add duplicate song via menus" /></p> <p><img src="/img/spotify_add_double_warning.jpg" alt="Duplicate warning" /></p> <p>However, what do you expect to happen if you press "Enter", when "Don't add" is highlighted?</p> <p><img src="/img/spotify_add_double_result.jpg" alt="Duplicate result on enter: added to playlist" /></p> <p>Exactly.</p> <h3 id="when-desktop-and-mobile-don-t-match">When desktop and mobile don't match</h3> <p>Spotify on desktop, and Spotify on mobile look share much of the visual design elements. I think the designers did a good job there. But as always, the devil is in the details.</p> <p>It is the features, or lack thereof where pain points arise. Spotify on desktop is usually a more powerful platform. This is understandable since the mouse and the large screen allow for more precise interactions than touch and a portrait mode phone screen.</p> <p>Still, after all these years, I still can't create new folders on mobile. It sucks that I need to open my desktop app for that.</p> <h2 id="final-words">Final words</h2> <p>I get it. I'm unlikely to be the average Spotify user. I've more than 300 playlists, rarely listen directly to albums or artists, and use this odd playlist centric workflow. Still, I have this hope, the hope of a long time subscriber, that one day the Spotify UI designers will recognize the power of user agency in app workflows and appreciate consist ordering more, like I do.</p> Github's malicious notification problem 2025-11-04T00:00:00+00:00 2025-11-04T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/github-malicious-notifications-copy/ <p>In the past four days or so, I've had notifications popping up for repositories which I didn't subscribe to myself.</p> <p>At first I thought the notification bubble just wouldn't disappear because of a little bug in <a href="https://github.com/refined-github/refined-github">Refined GitHub</a>, but on closer inspection (and disabling the extension to be sure), it turned out it wasn't so.</p> <p>This is what it looks like in my notifications list:</p> <p><img src="/img/github_malicious_notifications_unread.png" alt="List of unread malicious notifications on GitHub" /></p> <p>And because these notification aren't marked as read, the notification bubble at the top consistently suggests there's something new:</p> <p><img src="/img/github_malicious_notifications_icon.png" alt="Notification bubble due to malicious undismissable notifications on GitHub" /></p> <p>I wrote "aren't marked as read", but in reality the right word would be "can't". That is, the first image also shows that once you filter on the repository, while the bubble suggests there is one notification, according to the UI, there is none.</p> <h2 id="how">How</h2> <p>In GitHub you can tag someone by using <code>@username</code> syntax, which is useful in good faith situations. I presume spammers created some issues in their repository and started tagging lots of users.</p> <p>Presumably the repositories and accounts were removed by GitHub staff as manually navigating to the repository on GitHub, shows 404 pages.</p> <p><img src="/img/github_malicious_notifications_repo_404.png" alt="Removed repositories used by the spammers" /></p> <p>I'm sad that the notifications weren't removed too, and we now have to resort to alternative methods to remove the notifications.</p> <p>... Further searching suggests that this issue was initially <a href="https://github.com/orgs/community/discussions/6874">reported</a> on 28th of October, 2021, so it isn't a new problem.</p> <h2 id="workarounds">Workarounds</h2> <p><a href="https://github.com/orgs/community/discussions/174283">As</a> <a href="https://github.com/orgs/community/discussions/178439">it</a> turns out, I wasn't quite the only one who was affected by these spammers. In my searching, I found many community threads reporting the same issue, and asking for solutions to remove the "ghost" notifications.</p> <p>For example, in <a href="https://github.com/orgs/community/discussions/174246#discussioncomment-14471133">this</a> community thread, someone suggested to mark the notification as <em>read</em> using the GitHub API, which can be done using GitHub's own <code>gh</code> CLI tool (replace the <code>last_read_at</code> date as required):</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">gh</span><span> api notifications</span><span style="color:#bf616a;"> -X</span><span> PUT</span><span style="color:#bf616a;"> -F</span><span> last_read_at=2025-11-04T00:00:00Z </span></code></pre> <p>However, you should know that it doesn't quite remove the malicious notifications from your notifications dashboard:</p> <p><img src="/img/github_malicious_notifications_read.png" alt="List of read malicious notifications on GitHub" /></p> <p>Apparently, that can be done with the API too. I've adapted the invocation from <a href="https://github.com/orgs/community/discussions/178439">this</a> thread, to match on the repository owner (instead of the notification subject title), and added a flag to paginate the notifications in case you have more than one page like me:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">gh</span><span> api notifications</span><span style="color:#96b5b4;">\?</span><span>all=true | </span><span style="color:#bf616a;">jq -r </span><span>&#39;</span><span style="color:#a3be8c;">.[] | select(.repository.owner.login | test(&quot;(kamino-network|gitcoincom|ycombiinator|paradigm-notification)&quot;; &quot;i&quot;)) | .id</span><span>&#39; \ </span><span>| </span><span style="color:#bf616a;">xargs -L1 -I</span><span>{} gh api</span><span style="color:#bf616a;"> --method</span><span> DELETE \ </span><span style="color:#bf616a;"> -H </span><span>&quot;</span><span style="color:#a3be8c;">Accept: application/vnd.github+json</span><span>&quot; \ </span><span style="color:#bf616a;"> -H </span><span>&quot;</span><span style="color:#a3be8c;">X-GitHub-Api-Version: 2022-11-28</span><span>&quot; \ </span><span> /notifications/threads/{} </span></code></pre> <p>Step by step explained in case you too want to adapt the query (I did use the <code>gh</code> CLI, but you can send HTTP requests directly too).</p> <p>First we get all notifications by walking through all pages:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">gh</span><span> api notifications</span><span style="color:#96b5b4;">\?</span><span>all=true</span><span style="color:#bf616a;"> --paginate</span><span>` </span></code></pre> <p>Then we use <code>jq</code> to query the JSON response from the GitHub API and filter specifically on the spam GitHub account names <code>kamino-network</code>, <code>ycombiinator</code>, <code>gitcoincom</code>, and <code>paradigm-notification</code>. Return the <code>.id</code> of the notification if it's a match.</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">jq -r </span><span>&#39;</span><span style="color:#a3be8c;">.[] | select(.repository.owner.login | test(&quot;(kamino-network|ycombiinator|gitcoincom|paradigm-notification)&quot;)) | .id</span><span>&#39; </span></code></pre> <p>Combined so far with output:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">gh</span><span> api notifications</span><span style="color:#96b5b4;">\?</span><span>all=true</span><span style="color:#bf616a;"> --paginate </span><span>| </span><span style="color:#bf616a;">jq -r </span><span>&#39;</span><span style="color:#a3be8c;">.[] | select(.repository.owner.login | test(&quot;(kamino-network|ycombiinator|gitcoincom|paradigm-notification)&quot;)) | .id</span><span>&#39; </span><span> </span><span style="color:#bf616a;">19167019306 </span><span style="color:#bf616a;">19143656043 </span><span style="color:#bf616a;">19133393664 </span><span style="color:#bf616a;">19073885018 </span></code></pre> <p>Now we want to remove each of these notifications. To just remove one we can just substitute one of these id's:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">gh</span><span> api</span><span style="color:#bf616a;"> --method</span><span> DELETE \ </span><span style="color:#bf616a;"> -H </span><span>&quot;</span><span style="color:#a3be8c;">Accept: application/vnd.github+json</span><span>&quot; \ </span><span style="color:#bf616a;"> -H </span><span>&quot;</span><span style="color:#a3be8c;">X-GitHub-Api-Version: 2022-11-28</span><span>&quot; \ </span><span> /notifications/threads/19167019306 </span></code></pre> <p>We can use <code>xargs</code> to do this for all id's, resulting in the suggested query:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span style="color:#bf616a;">gh</span><span> api notifications</span><span style="color:#96b5b4;">\?</span><span>all=true | </span><span style="color:#bf616a;">jq -r </span><span>&#39;</span><span style="color:#a3be8c;">.[] | select(.repository.owner.login | test(&quot;(kamino-network|gitcoincom|ycombiinator|paradigm-notification)&quot;; &quot;i&quot;)) | .id</span><span>&#39; \ </span><span>| </span><span style="color:#bf616a;">xargs -L1 -I</span><span>{} gh api</span><span style="color:#bf616a;"> --method</span><span> DELETE \ </span><span style="color:#bf616a;"> -H </span><span>&quot;</span><span style="color:#a3be8c;">Accept: application/vnd.github+json</span><span>&quot; \ </span><span style="color:#bf616a;"> -H </span><span>&quot;</span><span style="color:#a3be8c;">X-GitHub-Api-Version: 2022-11-28</span><span>&quot; \ </span><span> /notifications/threads/{} </span></code></pre> <p><img src="/img/github_malicious_notifications_removed.png" alt="List of read malicious notifications on GitHub" /></p> <p>Tada!</p> Where are the security advisories of the recently compromised NPM packages? 2025-09-13T00:00:00+00:00 2025-09-13T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/finding-security-advisories/ <p>Four days ago, <a href="https://www.aikido.dev/blog/npm-debug-and-chalk-packages-compromised">news</a> came in that several packages on NPM were compromised; later it turned out to that it wasn't just one NPM account that was compromised by the (it seems) the same phishing attack, but multiple. The compromised accounts published new package versions containing malware which intercepted network traffic and replaced crypto wallet addresses with alternative malicious addresses. Various popular packages were affected, such as <code>chalk</code> and <code>debug-js</code> via <a href="https://www.npmjs.com/~qix">Qix</a>'s NPM account, and <code>duckdb</code> via the <a href="https://www.npmjs.com/~duckdb_admin">duckdb_admin</a> NPM account.</p> <p>The malware and compromised accounts (at least the one's we know of) was fairly quickly detected and actions were taken over the next few days to prevent the malicious NPM packages from being installed by downstream users. Props to all involved in the effort, and the open communication!</p> <h2 id="finding-the-security-advisories">Finding the security advisories</h2> <p>Now yesterday, I wanted to find the security advisories for the affected packages for a report at my day job. This turned out to be not as easy as I expected. Simply searching for, e.g. <a href="https://github.com/advisories?query=chalk">chalk</a> on the GitHub Advisory Database page gave me four results, <a href="https://github.com/prebid/Prebid.js/security/advisories/GHSA-jwq7-6j4r-2f92">three</a> <a href="https://github.com/advisories/GHSA-m662-56rj-8fmm">of</a> <a href="https://github.com/advisories/GHSA-w62p-hx95-gf2c">which</a> were sort of related, but none of them was the security issue in Chalk itself. I thought for a moment they maybe would have issued only a "top level" kind of advisory linked to e.g. the <code>debug</code> package, since that repo was used as a central location to file most of the <a href="https://github.com/debug-js/debug/issues/1005#issuecomment-3266885191">responses</a>, but <a href="https://github.com/advisories?page=1&amp;query=debug">no luck</a> (this sometimes happens when there is a CVE which links multiple packages, like in the case of the compromised <a href="https://github.com/advisories/GHSA-w62p-hx95-gf2c">DuckDB</a> packages. Changing the type to <a href="https://github.com/advisories?query=type%3Aunreviewed%20chalk">unreviewed</a> in case they would be filtered out by default gave even less results (0).</p> <p><img src="/img/security-advisories-chalk.png" alt="Result of searching for &#39;chalk&#39; on GitHub&#39;s security advisory page" /></p> <p>Then I tried searching via the repo's themselves, for example <a href="https://github.com/chalk/chalk/security/advisories">chalk</a> and <a href="https://github.com/debug-js/debug/security/advisories">debug</a>. These seem to be exclusive to information posed by the maintainers, which in this case opted to use different channels as there is nothing there.</p> <p>However, good news! They do exist! I have no idea how I should have found them without going through GitHub's own <a href="https://github.com/github/advisory-database">advisory-database</a> repo though, which is not exactly setup to be searched by humans directly. This repo also hosts <a href="https://github.com/github/advisory-database/issues/6099">these</a> <a href="https://github.com/github/advisory-database/issues/6103">two</a> related issues, which both contain partial lists with links to the specific advisories.</p> <p>As examples, here are the advisories for:</p> <ol> <li><a href="https://github.com/advisories/GHSA-2v46-p5h4-248w">chalk</a>, and</li> <li><a href="https://github.com/advisories/GHSA-8mgj-vmr8-frr6">debug</a></li> </ol> <p>If I have to guess, then I would guess the advisories only show up after package maintainers have approved it from a security dashboard; but I am not sure.</p> <p>Now, another reason why the search may be so difficult, is that there doesn't seem to be a CVE for most packages (I couldn't find one, for example <a href="https://nvd.nist.gov/vuln/search#/nvd/home?keyword=chalk&amp;resultType=records">chalk</a> again). DuckDB however does have <a href="https://nvd.nist.gov/vuln/search#/nvd/home?keyword=duckdb&amp;resultType=records">one</a>, and prebid-js <a href="https://nvd.nist.gov/vuln/detail/CVE-2025-59038">has</a> <a href="https://nvd.nist.gov/vuln/detail/CVE-2025-59039">two</a>.</p> <p>I'm unfamiliar with the process and can see that there are pratical limitations, not only because there are many parties involved and distributed over the world, but also because there are many different places where relevant info is posted, and a reasonable way to link from a root page may not be that obvious.</p> <p>Still, I feel that the way the information is published, presented and made searchable can be improved, for example by giving maintainers a more central way to publish status updates (i.e. written text) for multiple packages at once.</p> <p>Next time I would go directly to the <code>github/advisory-database</code> repository, as it <del>is unclear to me why some bits of information are not shown from the Advisories page</del>. While writing the post I found that GitHub's security advisories <a href="https://docs.github.com/en/code-security/security-advisories/working-with-global-security-advisories-from-the-github-advisory-database/about-the-github-advisory-database#malware-advisories">documentation</a> are probably hidden under explicit use of the <code>type:malware</code> tag by default, because "most of the vulnerabilities cannot be resolved by downstream users". This is, I assume, also why DuckDB's advisory does show up; it is not tagged under malware (but posted to DuckDB's <a href="https://github.com/duckdb/duckdb-node/security/advisories/GHSA-w62p-hx95-gf2c">duckdb-node</a> repo, and also has a CVE, i.e. CVE-2025-59037).</p> <p>I don't know yet whether hiding malware typed advisories by default is a practice I like. I definitely don't like that I this is non-obvious from the advisories page itself.</p> <p>Still, most of my critique stands: finding the right information fast is difficult, and the way advisories are posted is different per package.</p> <h2 id="when-npm-deletes-packages">When NPM deletes packages</h2> <p>The second thing I felt can be improved is the way [npmjs.org] removes affected packages. I consider it a good practice to remove the packages to reduce amount of affected downstream users. However, from the website, it is not clear why the package was removed (or whether it existed ever).</p> <p>Take for example <a href="https://www.npmjs.com/package/chalk?activeTab=versions">chalk</a>.</p> <p>The NPM website currently lists the following version history (I'll only list the last few packages):</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Version Downloads (Last 7 Days) Published </span><span>5.6.2 5,188,950 4 days ago </span><span>5.6.0 8,711,608 a month ago </span><span>5.5.0 1,769,950 a month ago </span><span>5.4.1 12,873,396 9 months ago </span><span>5.4.0 64,837 9 months ago </span></code></pre> <p>NB: It doesn't matter whether you tick the "show deprecated versions" box.</p> <p>From this page, you have no idea that 5.6.1 was ever published and subsequently removed.</p> <p>I think it would be an improvement to show the version, and mark it explicitly as "removed", so users are not left guessing. That would help you in your search to find context about why it was removed. More information like related advisories would be a bonus.</p> <p>Luckily, you will find the reason for the removal reported through the CLI, which is a good thing. However, if you use e.g. renovate bot (the open source version), you could have merged a PR to update the package without ever having installed the package yourself (and subsequently not have seen the notice), perhaps even because the PR was created before the infected package was noticed by anyone. It is that easy to end up with a deployed release affected with malware.</p> <p>My critiques for both these issues are quite the same: the way information is presented when it is critical is not yet where it could be. Of course there are many vendors which offer scanners for packages and package ecosystems, which maybe offer some central way to access information about the compromise. Still, sometimes you also need more specific or end-use information, if possible from the source. I hope that the future will bring better tooling or presentations to quickly assess vulnerability situations, even before a post-mortem took place.</p> Grammar checking from the CLI with Harper 2025-08-17T00:00:00+00:00 2025-08-17T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/harper-cli/ <p>A few months ago I learned of the existence of <a href="https://writewithharper.com">Harper</a>, which describes itself as a lightweight, offline, grammar checker. Both the website and GitHub <a href="https://github.com/automattic/harper">repository</a> are a bit sparse on how to use it, but they do link to all sorts of <a href="https://writewithharper.com/docs/integrations/language-server">integrations</a>, which I think all use the Harper language server. There is also <a href="https://writewithharper.com/docs/harperjs/introduction">harper.js</a>, mostly for web applications and browser extensions.</p> <p>While the language server (or <code>harper.js</code>) is great for extensive editor use, I sometimes just want to quickly check a file for spelling and grammatical issues. As it turns out, this is already possible by using the <code>harper-cli</code> tool.</p> <p>This CLI is not documented on the documentation website, probably, as I found out later, because it is still listed as an <a href="https://github.com/Automattic/harper/tree/c37fa1a437279ed8449f75a80178c55f29e4df80/harper-cli">experimental</a> frontend. I'm glad it exists though; it saved me from even considering how to use the LSP from the command line directly, and from installing and using one of the integrations.</p> <h2 id="installing-it">Installing it</h2> <p>Installing it is easy! If you have <a href="https://www.rust-lang.org/">Rust</a> and <a href="https://doc.rust-lang.org/cargo/">Cargo</a> installed, you can install it from source by running <code>cargo install --locked --git https://github.com/Automattic/harper.git harper-cli</code>.</p> <p>On Windows I also found out you can install it using <a href="https://scoop.sh/">scoop</a>. I was delighted to see that <code>scoop install harper</code> didn't just install the language server binary, but also <code>harper-cli</code>.</p> <h2 id="using-it">Using it</h2> <p>The current version (<code>0.1.0</code>), shows the following help page when running <code>harper-cli --help</code>:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>A debugging tool for the Harper grammar checker </span><span> </span><span>Usage: harper-cli.exe &lt;COMMAND&gt; </span><span> </span><span>Commands: </span><span> lint Lint a provided document </span><span> parse Parse a provided document and print the detected symbols </span><span> spans Parse a provided document and show the spans of the detected tokens </span><span> annotate-tokens Parse a provided document and annotate its tokens </span><span> metadata Get the metadata associated with a particular word </span><span> forms Get all the forms of a word using the affixes </span><span> words Emit a decompressed, line-separated list of the words in Harper&#39;s dictionary </span><span> summarize-lint-record Summarize a lint record </span><span> config Print the default config with descriptions </span><span> mine-words Print a list of all the words in a document, sorted by frequency </span><span> core-version Print harper-core version </span><span> rename-flag Rename a flag in the dictionary and affixes </span><span> compounds Emit a decompressed, line-separated list of the compounds in Harper&#39;s dictionary. As long as there&#39;s either an open or hyphenated spelling </span><span> case-variants Emit a decompressed, line-separated list of the words in Harper&#39;s dictionary which occur in more than one lettercase variant </span><span> nominal-phrases Provided a sentence or phrase, emit a list of each noun phrase contained within </span><span> help Print this message or the help of the given subcommand(s) </span><span> </span><span>Options: </span><span> -h, --help Print help </span><span> -V, --version Print version </span></code></pre> <p>For my use case, the most simple command sufficed:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$ harper-cli lint .\content\posts\2025-05-15_thank_you_all_for_ten_years_of_stable_rust.md </span></code></pre> <p>That gave me the following result:</p> <p><img src="/img/harper-cli.png" alt="Result of harper-cli output showing some grammatical errors" /></p> <p>What a delightful way to check for flagrant spelling errors in markdown files. Thanks Harper authors!</p> Thank you all for 10 years of (stable) Rust 2025-05-15T00:00:00+00:00 2025-05-15T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/thank-you-all-for-ten-years-of-stable-rust/ <p>Today marks the 10th anniversary of <a href="https://blog.rust-lang.org/2015/05/15/Rust-1.0/">Rust 1.0</a>. <a href="https://internals.rust-lang.org/t/rust-1-87-0-pre-release-testing/22896">Rust 1.87</a> will be released live on stage at the <a href="https://rustweek.org/celebration/">10 years of Rust celebration</a> this afternoon (hosted by RustWeek, there are still tickets I heard). Over these past 10 years, Rust had a major positive impact on my life in various ways, and I'm extremely grateful for that.</p> <p><strong>Thanks to all Rust contributors and to our community, it has been a blast 🎉</strong>.</p> <p>An anniversary is usually a good moment to look forward, but it also marks a good moment to take a step back, and reflect a bit on the past. Yesterday I scrolled back through the <a href="https://github.com/rust-lang/rust/blob/master/RELEASES.md">release notes</a> for a bit, and reflected on some improvements which I've been taking more and more for granted.</p> <p>The one which pops out of that list the most (for me) is probably the <code>?</code> operator. It was stabilized in <a href="https://blog.rust-lang.org/2016/11/10/Rust-1.13/">Rust 1.13.0</a>, but prototyped via the <code>try!</code> macro some time before that. I can truthfully say that every time I write in another language, I miss that <code>?</code> and the associated <a href="https://doc.rust-lang.org/std/ops/trait.Try.html"><code>Try</code></a> trait (I kind of wish the trait was marked stable 😅).</p> <!-- The other big ones which made Rust less cumbersome to write were of course [non lexical lifetimes](https://blog.rust-lang.org/2018/12/06/Rust-1.31-and-rust-2018/#non-lexical-lifetimes), lifetime elision improvements, the deref improvements (no more `&***`) and a boat load of library improvements. <sup>...and then there is of course the documentation, the tools, the community, the transparent and in the open development. And so much more.</sup> --> <p>The time where a large portion of the community was on nightly by default is now far in the past. With the vision doc for Rust taking shape, and likely hundreds of other <a href="https://rust-lang.github.io/rust-project-goals/2025h1/index.html">improvements</a>, I can't wait for the next 10 years of stable Rust.</p> Accessibility and Rust podcast at RustWeek 2025-05-13T00:00:00+00:00 2025-05-13T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/rustweek-accessibility-and-rust-podcast/ <p>Just hours ago, the first full conference day of <a href="https://rustweek.org/">RustWeek 2025</a> ended. Everything went smoothly, so big props to the organisers, volunteers*, the audio and video crew, and staff at the venue. Also shoutout to the people who designed and organized the stickers, they're really really great, and also to the excellent barista coffee ☕.</p> <p><img src="/img/stickers_rustweek.jpg" alt="A small sample of stickers at RustWeek" /></p> <p>The day ended for me by attending the live recording of the <a href="https://corrode.dev/podcast">Rust in Production</a> podcast in which Matthias Endler interviewed Niko Matsakis about his experiences of building Rust over the past 10 years, and by attending the live recording of another podcast titled 'Accessibility and Rust' (which if I recall correctly will be published on the <a href="https://rustacean-station.org">Rustacean Station</a>). In this podcast <a href="https://github.com/luukvanderduim">Luuk van der Duim</a> was joined by <a href="https://github.com/mwcampbell">Matt Campbell</a> and <a href="https://github.com/DataTriny">Arnold Loubriat</a> to talk about their experience of working on libraries and tooling to make graphical user interfaces (GUI's) more accessible. Both sessions were excellent. I wanted to take a moment to say a few words about the second one, which really spoke to me.</p> <p>During the podcast Matt and Arnold give some insight into how they "see" graphical computer programs, and also how they often can't because of shortcomings or lack of accessibility in programs. From a software engineering point of view, I have a feeling, we often shove accessibility features under the rug because of a perceived lack of business need, cost or complexity, so I'm happy there are people working on lowering the barrier.</p> <p>This talk also addressed another possible reason (amplified by the audience after the recording session), that for many developers who don't use assistive technologies like screenreaders, it's opaque how to build software that is accessible. On top of that, there is also the question of how to test whether you did it right, given you're not a regular user of assistive technologies.</p> <p><a href="https://github.com/AccessKit/accesskit">AccessKit</a> tries to help solve the first question, and has amongst others, been used to improve accessibility support for <a href="https://github.com/slint-ui/slint/blob/9b176ffb17fcdd33b2e16c70f07d7083228bdab2/internal/backends/winit/accesskit.rs#L97">Slint</a>, a GUI toolkit**.</p> <p>For the second question, I hope people will support their work and perhaps provide funding to them (or others) to develop a general test suite or framework which can be used by implementers of accessibility libraries***. In the mean time, I found that there is an <a href="https://developer.mozilla.org/en-US/docs/Learn_web_development/Core/Accessibility">introduction</a> on the topic over at MDN, which has a particular focus on accessibility for webpages, but also includes a more general introduction.</p> <p>Matt will give a talk about <a href="https://gribnau.dev/posts/rustweek-accessibility-and-rust-podcast/AccessKit">AccessKit</a> tomorrow at <a href="https://time.is/compare/1025_14_May_2025_in_Utrecht">10:25</a> local time (main track). RustWeek has a <a href="https://rustweek.org/live/wednesday/">livestream</a> if you want to follow along online. Recordings of talks will be published online at a later moment.</p> <p><em>* Disclaimer: I also volunteered, props go to the other volunteers 😉.</em> <br> <em>** I'm not involved with Slint in any way</em> <br> <em>*** I may be ignorant of the existence of such a test suite; sorry!</em></p> Orphaned markdown [brackets] 2025-04-19T00:00:00+00:00 2025-04-19T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/forgotten-brackets/ <p>Every project needs a changelog. And let's be honest, the work <a href="https://keepachangelog.com/en/1.1.0/">keep-a-changelog</a> did to make this a more common practice is awesome. But there is something many seem to forget: Did you ever notice [version numbers] in release titles within a <code>CHANGELOG.md</code>? These brackets were supposed to be <a href="https://www.markdownguide.org/basic-syntax/#reference-style-links">links</a>. But in reference style, you need to complete the reference elsewhere. Let's look at an example (also in the <a href="https://github.com/mullvad/mullvadvpn-app/blob/6d7b4ba7bdc497093dbd5861a8a6a4842574e6f7/CHANGELOG.md">wild</a>):</p> <pre data-lang="markdown" style="background-color:#2b303b;color:#c0c5ce;" class="language-markdown "><code class="language-markdown" data-lang="markdown"><span style="color:#8fa1b3;">## [0.18.3] - 2025-04-19 </span><span> </span><span style="color:#bf616a;">- Lost: reference to the version number 0.18.3 somewhere </span></code></pre> <p>This snippet, produces the following output:</p> <hr /> <h2 id="0-18-3-2025-04-19">[0.18.3] - 2025-04-19</h2> <ul> <li>Lost: references to version numbers</li> </ul> <hr /> <p>However, if you scroll to the bottom of the keep-a-changelog <a href="https://raw.githubusercontent.com/olivierlacan/keep-a-changelog/refs/heads/main/CHANGELOG.md">example</a>, you can see that the version numbers in brackets, have their link completed at the bottom. And rendered, it looks like <a href="https://github.com/olivierlacan/keep-a-changelog/blob/main/CHANGELOG.md">this</a>.</p> <p>So how do we fix our changelog? By adding reference style links:</p> <pre data-lang="markdown" style="background-color:#2b303b;color:#c0c5ce;" class="language-markdown "><code class="language-markdown" data-lang="markdown"><span style="color:#8fa1b3;">## [0.18.4] - 2025-04-19 </span><span> </span><span style="color:#bf616a;">- Found: references to version numbers </span><span style="color:#bf616a;"> </span><span style="color:#d08770;">[0.18.4]: https://github.com/foresterre/cargo-msrv/compare/v0.18.3...v0.18.4 </span></code></pre> <p>This produces the following output:</p> <hr /> <h2 id="0-18-4-2025-04-19"><a href="https://github.com/foresterre/cargo-msrv/compare/v0.18.3...v0.18.4">0.18.4</a> - 2025-04-19</h2> <ul> <li>Found: references to version numbers</li> </ul> <hr /> <h1 id="don-t-like-linking-to-the-diff">Don't like linking to the diff?</h1> <p>Not a problem. Use an alternative like a <a href="https://github.com/foresterre/cargo-msrv/releases/tag/v0.18.4">release</a> page or something else you like. Or just leave the brackets out:</p> <hr /> <h2 id="0-18-5-2025-04-19">0.18.5 - 2025-04-19</h2> <ul> <li>Also good: This version number has no link</li> </ul> <hr /> <h1 id="call-to-action">Call to action</h1> <p>So next time you write a changelog, please keep these digital tumbleweeds out. Complete your links, fix your [brackets]!</p> A sneak peek Into::<Rust> (slide deck, 2022) 2025-02-27T00:00:00+00:00 2025-02-27T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/rust-slides/ <p>Today I rediscovered a slide deck I made for a lunch talk I gave at the beginning of 2022. It is an introduction to Rust, and specifically the features which made me fall in love with Rust since 2014.</p> <p>It was aimed at an engineering audience consisting of among others software engineers, mechanical engineers and control systems engineers. Everyone in the audience had at least some programming experience.</p> <p>I decided to post it to my blog, in case anybody would like to use it for inspiration, or would find it otherwise useful.</p> <p>The slide deck can be found <a href="/pdf/sneak_peek_into_rust.pdf">here</a>.</p> <p>P.S. the title was a bit of an inside joke. It might not roll off the tongue as nicely as I would have wanted otherwise 🙃.</p> A few cargo-msrv 0.16 release highlights 2024-10-09T00:00:00+00:00 2024-10-09T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/cargo-msrv-0-16/ <p>A quick tour through some of the <a href="https://github.com/foresterre/cargo-msrv/releases/tag/v0.16.0"><code>cargo msrv 0.16.0</code></a> highlights.</p> <h2 id="what-is-cargo-msrv">What is <a href="https://github.com/foresterre/cargo-msrv"><code>cargo-msrv</code></a>?</h2> <p><code>cargo-msrv</code> is a <a href="https://crates.io/categories/development-tools::cargo-plugins">cargo plugin</a> which helps you "Find the minimum supported Rust version (MSRV) for your project".</p> <p>Aside from finding the MSRV, it has additional tools baked in, such as listing the MSRV's of dependencies and verifying your (new) code against a given MSRV.</p> <p>Find out more <a href="https://github.com/foresterre/cargo-msrv">here</a> 😉.</p> <h2 id="the-tour">The tour</h2> <p>The remainder of this post can also be found in the <code>cargo-msrv</code> <a href="https://foresterre.github.io/cargo-msrv/releases/v0.15_v0.16_highlights.html">book</a>.</p> <h3 id="the-noticeable-one-cargo-msrv-find">The noticeable one - cargo msrv find</h3> <p>The tagline of <code>cargo-msrv</code> is "Find the minimum supported Rust version (MSRV) for your project". Previously, one could achieve this by running <code>cargo msrv</code>. If you want to do the same in 0.16, you instead should run <code>cargo msrv find</code>. The top level <code>cargo msrv</code> action is no more.</p> <p>There are two primary reasons to move this action to a subcommand instead of keeping it at the top level:</p> <ol> <li>Consistency: <code>cargo msrv</code> can do more than that tagline, and placing all actions on the subcommand level signals that they're equals.</li> <li>Unsupported CLI flags and options: When actions are placed on two layers, and one of these layers is below the other, then the bottom layer inherits its flags and options, even though they do not always overlap. For example, the set of CLI flags and options of <code>cargo msrv find</code> and <code>cargo msrv list</code> are not identical. <code>cargo msrv find</code> for example has an option called <code>--release-source</code> which should be present for <code>cargo msrv find</code> but not for <code>cargo msrv list</code>. If <code>cargo msrv find</code> would still be run as <code>cargo msrv</code>, you could also invoke this option for <code>cargo msrv list</code>, like so: <code>cargo msrv --release-source rust-dist list</code>. However, contextually, the <code>--release-source</code> option does not make sense for <code>cargo msrv list</code>, so previously it was ignored. By making <code>cargo msrv find</code> a subcommand like <code>cargo msrv list</code>, the flags and options which are not shared between all actions can be put solely below their own subcommand.</li> </ol> <p>A consequence of (2) is that some unnecessary options and flags have been removed from the top level, and so this is a breaking change, not just for <code>cargo msrv find</code> but also for <code>cargo msrv list</code>.</p> <p>Minor reasons for this change include that I can now talk about <code>cargo msrv find</code> as "<code>cargo msrv find</code>" instead of <code>cargo msrv (find)</code> or "the top level command". Plus, it addressed some difficulties around the code which does CLI parsing, about which I wrote in <a href="https://gribnau.dev/posts/puzzle-sharing-declarative-args-between-top-level-and-subcommand">this</a> previous post. I'm glad I opted to go this route instead though 😅.</p> <h3 id="the-ui-part-1-output-format-options">The UI part 1: output format options</h3> <p>The way the UI is rendered has been updated. Internally, it is now easier to add and maintain different output formats.</p> <p><code>cargo-msrv</code> now supports 3 output formats:</p> <ul> <li>human (the default one, intended for the human eye)</li> <li>json (intended for machine readability)</li> <li>minimal (*new*, it was requested for environments where people only care about success/failure, such as CI)</li> </ul> <h3 id="the-ui-part-2-cargo-msrv-find-and-verify-human-output">The UI part 2: <code>cargo msrv find</code> and <code>verify</code> "human" output</h3> <p>As they say, "a picture is a thousand words":</p> <p><strong>Previously...</strong></p> <p><a href="https://asciinema.org/a/465459"><img src="https://asciinema.org/a/465459.svg" alt="asciicast" /></a></p> <p><strong>New...</strong></p> <p><a href="https://asciinema.org/a/JGyYmk7LNJvfDrc2oMQEt0SFF"><img src="https://asciinema.org/a/JGyYmk7LNJvfDrc2oMQEt0SFF.svg" alt="asciicast" /></a></p> <p>I'll be iterating the UI further in the future. Constructive feedback is more than welcome!</p> <h3 id="cargo-msrv-find-write-msrv"><code>cargo msrv find --write-msrv</code></h3> <p>This option will write the MSRV to your Cargo manifest:</p> <p><a href="https://asciinema.org/a/679863?t=47"><img src="https://asciinema.org/a/679863.svg" alt="asciicast" /></a></p> <h3 id="cargo-msrv-find-min-and-max"><code>cargo msrv find --min</code> and <code>--max</code></h3> <p>The <code>--min</code> and <code>--max</code> options would previously only take three component semver versions like "1.2.3" or editions. It is common to specify the MSRV in a two component version like "1.2", so these are now also supported.</p> <h3 id="cargo-msrv-verify-rust-version"><code>cargo msrv verify --rust-version</code></h3> <p><code>cargo msrv verify</code> can be used to check whether your project is compatible with its MSRV. The MSRV is usually read from the Cargo manifest (<code>Cargo.toml</code>). Sometimes it can be useful to provide it manually instead. That's where this option comes in handy.</p> <p>It should be noted that <code>cargo-msrv</code> does, at present, not unset any value you may have specified in the Cargo manifest. So if you have a Cargo manifest with <code>rust-version = "1.56.0"</code> and supply the <code>--rust-version</code> option with the value <code>1.55.0</code>, the cargo project will (if the default options are used) fail to compile, and as a consequence <code>cargo-msrv</code> will report that your crate is not compatible with the specified MSRV.</p> <h3 id="fetching-the-rust-releases-index">Fetching the rust releases index</h3> <p>The rust releases index, the thing we use to figure out which Rust versions exist, are now only fetched when a subcommand needs it (currently <code>cargo msrv find</code> and <code>cargo msrv verify</code>).</p> <h3 id="the-changelog">The changelog</h3> <p>The complete changelog can be found <a href="https://github.com/foresterre/cargo-msrv/blob/v0.16.0/CHANGELOG.md">here</a>.</p> <h2 id="thanks">Thanks!</h2> <p>Thanks to all contributors, whether you submitted a <a href="https://github.com/foresterre/cargo-msrv/pulls">PR</a> or reported an <a href="https://github.com/foresterre/cargo-msrv/issues">issue</a>, or contributed in some other way.</p> <p>Some of your issues and PR's really made my day! 💛</p> Puzzle: Sharing declarative args between top level and subcommand using Clap 2024-06-25T00:00:00+00:00 2024-06-25T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/puzzle-sharing-declarative-args-between-top-level-and-subcommand/ <p>Alternative title: <em>... And a short introduction to <code>cargo msrv</code></em></p> <h1 id="the-problem">The problem</h1> <p>In <a href="https://github.com/foresterre/cargo-msrv">cargo-msrv</a> we have the following situation. The top level command is used when "finding the MSRV" of a Rust project, while a subcommand can be used to verify that a specific MSRV works for a given project.</p> <p>The former can be done by running <code>cargo msrv</code> while the latter would be <code>cargo msrv verify</code>.</p> <p>Once upon a time, an issue was reported that <code>cargo msrv --target x verify</code> worked, but <code>cargo msrv verify --target x</code> did not. Another Cargo tool used by the reporter, together with <code>cargo-msrv</code>, would always specify the latter form, so the former could not be used as a workaround.</p> <p>While diving into this issue I figured, this should work:</p> <ol> <li><code>cargo msrv --target x verify</code></li> <li><code>cargo msrv verify --target x</code> (equivalent to 2 for the user)</li> </ol> <p>But this should not:</p> <ul> <li><code>cargo msrv --target x verify --target y</code></li> </ul> <p>The latter form is confusing to a user. If you allow both, you have to consider which takes precedence. Or whether they define the same thing.</p> <p>This should also not work (since the <code>list</code> subcommand currently doesn't use the <code>target</code> CLI argument at all):</p> <ul> <li><code>cargo msrv --target x list</code></li> <li><code>cargo msrv list --target x</code></li> </ul> <p>This post describes a short journey into the search for a satisfying solution.</p> <p><code>cargo-msrv</code> uses <code>clap</code>, the most commonly used CLI argument parser (in the Rust library landscape). For the remainder of this post I will assume some familiarity with Rust and <code>clap</code>.</p> <h2 id="background-on-cargo-msrv">Background on cargo msrv</h2> <p>This section can be useful to better understand this post since I decided to keep the original problem, instead of describing a more minimal reproduction. But feel free to skip this section (I might refer back to it though 😜).</p> <p><strong>A short history</strong></p> <p><code>cargo-msrv</code> was originally born out of a desire to find the MSRV for a Rust project (more specifically package). MSRV stands for "minimal supported Rust version" and is the earliest or oldest version supported by a Rust project. For different projects this may mean different things, but for this post I will consider "support" as "does compile with a Rust toolchain of a certain version".</p> <p>Fast forward a few years, and the MSRV has become somewhat more ubiquitous which can also be seen by its inclusion into Cargo as the <a href="https://doc.rust-lang.org/cargo/reference/manifest.html#the-rust-version-field">rust-version</a>. Over time some additional tools were added to <code>cargo-msrv</code>. One of these was the <code>cargo msrv verify</code> subcommand.</p> <p>This subcommand can be used to check whether a Rust project supports its defined MSRV (e.g. via this <code>rust-version</code> field in the Cargo manifest). For example, in a CI pipeline you can use this to check whether your project works for the version you promised to your users.</p> <p>Originally, I kept the <code>cargo msrv</code> top level command aside from the subcommands for backwards compatibility reasons. In hindsight I probably shouldn't have done that, but as is, their coexistence at least provides me with the opportunity to write this blog post 😅.</p> <p><strong>How cargo msrv works</strong></p> <p>I described the "support" from "minimal supported Rust version" (MSRV) above as the somewhat simplified "does compile with a Rust toolchain of a certain version".</p> <p>You may write that as a function like so: <code>fn is_compatible(version) -&gt; bool</code>. If you run this test for some Rust version, when the function produces the value <code>true</code>, then we consider the Rust version to be supported. If instead the function produces the value <code>false</code>, then the Rust version is not supported.</p> <p><code>cargo msrv</code> specifically searches for the <em>minimal</em> Rust version which is supported by a given Rust project. While there are some caveats, we build upon Rust's <a href="https://blog.rust-lang.org/2014/10/30/Stability.html#committing-to-stability">stability promise</a> . In our case that is the idea that Rust versions are backwards compatible.</p> <p>For a simple example to determine an MSRV, you can linearly walk backwards from the most recent Rust version to the earliest. When your project doesn't compile for a specific Rust version, then the last version that did compile can be considered your MSRV.</p> <p>Let's make it a bit more concrete with an example. For this example, we assume that Rust the following Rust versions exist: <code>1.0.0</code> up to and including <code>1.5.0</code>.</p> <p>Consider a project which uses the <a href="https://doc.rust-lang.org/std/time/struct.Duration.html#">Duration</a> API which was stabilised by <a href="https://github.com/rust-lang/rust/blob/master/RELEASES.md#version-130-2015-09-17">Rust 1.3.0</a> (and nothing more recent 😉).</p> <p>Then, if you would compile this project with Rust version from most recent to least recent, you would expect the following to happen:</p> <ul> <li><code>is_compatible(Rust 1.5.0)</code> returns <code>true</code> ✅</li> <li><code>is_compatible(Rust 1.4.0)</code> returns <code>true</code> ✅</li> <li><code>is_compatible(Rust 1.3.0)</code> returns <code>true</code> ✅</li> <li><code>is_compatible(Rust 1.2.0)</code> returns <code>false</code> ❌ ("Duration is not stable")</li> <li><code>is_compatible(Rust 1.1.0)</code> returns <code>false</code> ❌</li> <li><code>is_compatible(Rust 1.0.0)</code> returns false ❌</li> </ul> <p>Since we only care about the <em>minimal</em> Rust version, you could have stopped searching after compiling Rust 1.2.0; Rust 1.3.0 was the earliest released Rust version which worked.</p> <p>In reality doing a linear search is quite slow (at the time of writing, there are 79 minor versions), so we primarily use a binary search instead to incrementally reduce the search space.</p> <p><code>cargo msrv verify</code> works quite similar to "finding the MSRV", but instead of running a search which produces as primary output the MSRV, in this case the MSRV is already known in advance. So given a <code>MSRV</code> of <code>1.3.0</code> we just run the <code>is_compatible(Rust 1.3.0)</code> function once. If it returns <code>true</code> we can say that the 1.3.0 is an acceptable MSRV (although not necessarily strictly so). More importantly, if it returns false, then the specified version is actually not supported, and thus can not be an MSRV).</p> <p>Enough background, back to the post.</p> <h1 id="definition-of-the-cli">Definition of the CLI</h1> <p><code>cargo-msrv</code> uses <code>clap</code> as its CLI argument parser. It nowadays uses the <code>macro derive</code> based API. In the code blocks below, I have isolated the primary definition of the CLI which describes the <code>cargo msrv</code> top level command (a.k.a. <code>Find</code> in the code), and the <code>cargo msrv verify</code> subcommand.</p> <p>I've taken the liberty to remove some unnecessary details, and added some arrows and comments for relevant items. Otherwise, this code is the copied directly from the actual source.</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span>#[</span><span style="color:#bf616a;">command</span><span>(version)] </span><span style="color:#b48ead;">pub struct </span><span>CargoMsrvOpts { </span><span style="color:#65737e;">// &lt;- The top level, i.e. `cargo msrv {...}` </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">find_opts</span><span>: FindOpts, </span><span style="color:#65737e;">// &lt;- Options relevant for &quot;find msrv&quot; </span><span> </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">shared_opts</span><span>: SharedOpts, </span><span> </span><span> #[</span><span style="color:#bf616a;">command</span><span>(subcommand)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">subcommand</span><span>: Option&lt;SubCommand&gt;, </span><span style="color:#65737e;">// &lt;- Subcommands, like &quot;verify&quot; in &quot;cargo msrv verify&quot; </span><span>} </span><span> </span><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span>#[</span><span style="color:#bf616a;">command</span><span>(next_help_heading = &quot;</span><span style="color:#a3be8c;">Find MSRV options</span><span>&quot;)] </span><span style="color:#b48ead;">pub struct </span><span>FindOpts { </span><span> #[</span><span style="color:#bf616a;">arg</span><span>(long, conflicts_with = &quot;</span><span style="color:#a3be8c;">linear</span><span>&quot;)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">bisect</span><span>: </span><span style="color:#b48ead;">bool</span><span>, </span><span> </span><span> #[</span><span style="color:#bf616a;">arg</span><span>(long, conflicts_with = &quot;</span><span style="color:#a3be8c;">bisect</span><span>&quot;)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">linear</span><span>: </span><span style="color:#b48ead;">bool</span><span>, </span><span> </span><span> </span><span style="color:#65737e;">// ... omitted some options for brevity </span><span> </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">runner_opts</span><span>: RunnerOptsFind, </span><span>} </span></code></pre> <p><code>CargoMsrvOpts</code> is the top level CLI interface, i.e. <code>cargo msrv</code>. The options relevant to "finding the MSRV" are specified on the top level (i.e. <code>FindOpts</code>).</p> <p><code>FindOpts</code> should not be used for other subcommands (ironic considering this post, I know, at least <code>cargo msrv verify --help</code> doesn't list them 😜).</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Subcommand)] </span><span>#[</span><span style="color:#bf616a;">command</span><span>(propagate_version = true)] </span><span style="color:#b48ead;">pub enum </span><span>SubCommand { </span><span> List(ListOpts), </span><span> Set(SetOpts), </span><span> Show, </span><span> Verify(VerifyOpts), </span><span style="color:#65737e;">// &lt;- The options for the verify subcommand </span><span>} </span><span> </span><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span>#[</span><span style="color:#bf616a;">command</span><span>(next_help_heading = &quot;</span><span style="color:#a3be8c;">Verify options</span><span>&quot;)] </span><span style="color:#b48ead;">pub struct </span><span>VerifyOpts { </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">runner_opts</span><span>: RunnerOpts, </span><span style="color:#65737e;">// &lt;- The options we want to share between the top level &quot;find msrv&quot; (i.e. &#39;cargo msrv&#39;) and &quot;verify msrv&quot; subcommand (i.e. &#39;cargo msrv verify&#39;) </span><span> </span><span> #[</span><span style="color:#bf616a;">arg</span><span>(long, value_name = &quot;</span><span style="color:#a3be8c;">rust-version</span><span>&quot;)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">rust_version</span><span>: Option&lt;BareVersion&gt;, </span><span>} </span></code></pre> <p>Options supplied to the <code>verify</code> subcommand are specified by the <code>VerifyOpts</code> struct. As a user, you would interface with it like so: <code>cargo msrv verify {opts}</code>, e.g. <code>cargo msrv verify --target example</code>.</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span style="color:#b48ead;">pub struct </span><span>RunnerOpts { </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">rust_releases_opts</span><span>: RustReleasesOpts, </span><span> </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">toolchain_opts</span><span>: ToolchainOpts, </span><span style="color:#65737e;">// &lt;- The examples will use this one </span><span> </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">cargo_check_opts</span><span>: CheckCommandOpts, </span><span>} </span><span> </span><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span>#[</span><span style="color:#bf616a;">command</span><span>(next_help_heading = &quot;</span><span style="color:#a3be8c;">Toolchain options</span><span>&quot;)] </span><span style="color:#b48ead;">pub struct </span><span>ToolchainOpts { </span><span> #[</span><span style="color:#bf616a;">arg</span><span>(long, value_name = &quot;</span><span style="color:#a3be8c;">TARGET</span><span>&quot;)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">target</span><span>: Option&lt;String&gt;, </span><span> </span><span> #[</span><span style="color:#bf616a;">arg</span><span>(long, value_name = &quot;</span><span style="color:#a3be8c;">COMPONENT</span><span>&quot;)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">add_component</span><span>: Vec&lt;String&gt;, </span><span>} </span></code></pre> <p><code>RunnerOpts</code> specifies options relevant to run the <code>is_compatible</code> test I introduced in the <em>How cargo msrv works</em> section. What this actually entails is not relevant to this post. All you need to know is that each of these flattened commands define some arguments. I will use the arguments in <code>ToolchainOpts</code> for the examples: <code>--target</code> (as my primary example) and <code>--add-component</code> (just to point out some edge cases).</p> <p>The last detail I will add is that for both the "find msrv" top level command and each of the subcommands, we produce a flattened context which contains the inputs for that command. Both the <code>FindContext</code> and <code>VerifyContext</code> contain a <code>pub toolchain: ToolchainContext</code> field. The context is currently created via the <code>TryFrom</code> trait from the <code>Opts</code> (or resolved from the environment) to the <code>Context</code>.</p> <p>The fields in the <code>ToolchainContext</code> can be considered static for the duration of the program.</p> <h1 id="the-goal">The goal</h1> <p>Let's make the problem a bit more concrete again.</p> <p>First lets be explicit: the problem is limited to "matching (in name and type) CLI arguments which can be provided to both <code>cargo msrv</code> and <code>cargo msrv verify</code>", and I'll mention that if no value is given for either, we use some default which we will assume just exists.</p> <p>In the end, (1) <strong>we want to end up with a CLI interface like <code>cargo msrv {top_level_arg} verify {subcommand_arg}</code> where a matching argument is provided to either the top level <code>xor</code> the subcommand <code>xor</code> use the default</strong>.</p> <p>In addition, (2) <strong>the matching args should not be available to subcommands, other than <code>verify</code></strong>.</p> <p>In the next section, I'm going to throw some ideas over the wall. To generate these ideas I primarily used the <code>clap</code> rustdoc documentation on <a href="https://gribnau.dev/posts/puzzle-sharing-declarative-args-between-top-level-and-subcommand/docs.rs/clap%60">docs.rs</a> as a reference.</p> <p>Let me spoil it for you in advance 🙄: None of these solutions really make me happy, so I will probably choose a pragmatic choice instead.</p> <p>I mention this beforehand because I want to explicitly state that none of this is something I blame on <code>clap</code> (or anyone else). <code>clap's</code> documentation is extensive, and is very well written. If it is described, then I didn't look in the right places. It might just be a puzzle I haven't solved yet 🧩.</p> <p>Also, this scenario where you have a top level CLI and some subcommands which have to share some things is most likely non standard, and honestly, discouraged for similar reasons as why I'm putting this effort in: user experience.</p> <h1 id="idea-1-merging-opts">Idea 1: merging Opts</h1> <p>A first idea is to keep the current definition and simply merge its values.</p> <p>You may see some obvious flaws with this: first, goal (2) will not be met, since one of the <code>RunnerOpts</code> is defined on <code>FindOpts</code> which is at the top level of the CLI, so subcommands other than <code>verify</code> will also have allow the arguments defined by <code>RunnerOpts</code> to be specified (even though they will be ignored). Second, goal (1) is dependent on custom merging logic.</p> <p>For this, let's quickly consider the output which we would get from <code>clap</code> after it parsed the CLI arguments like in this idea (in simplified Rust code), to consider what it would look like:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">let</span><span> opts = CargoMsrvOpts { ... }; </span><span style="color:#65737e;">// Output from Clap&#39;s parse method </span><span style="color:#b48ead;">let</span><span> find_opts = opts.find_opts; </span><span style="color:#b48ead;">let </span><span>Some(VerifyOpts(verify_opts)) = opts.subcommand </span><span style="color:#b48ead;">else </span><span>{ todo!() }; </span><span> </span><span style="color:#b48ead;">let</span><span> find_toolchain_opts = find_opts.runner_opts.toolchain_opts; </span><span style="color:#b48ead;">let</span><span> verify_toolchain_opts = verify_opts.runner_opts.toolchain_opts; </span></code></pre> <p>If <code>opts.subcommand</code> is not <code>Some(VerifyOpts(_))</code> then we can simply take all arguments from the <code>find_toolchain_opts</code>.</p> <p>If <code>opts.subcommand</code> is <code>Some(VerifyOpts(_))</code>, then we would need to merge <code>find_toolchain_opts</code> and <code>verify_toolchain_opts</code>.</p> <p>Most of this merging code would likely be fairly trivial considering the limited common CLI input options and the XOR-or-default condition that <code>clap</code> would enforce if this concept worked.</p> <blockquote> <p>Values which are marked as optional, i.e. via the <code>Option&lt;T&gt;</code> type are trivial. If one is <code>Some</code>, then the other one must be <code>None</code> (by the XOR-or-default condition).</p> <p>Values which instead use a default value, may not be so clear cut. There is the question: did the user provide this default value, or was it omitted. The first case is a choice, similar to <code>Some</code>, while the second case is a fallback to a default like <code>None</code>. This information is no longer known after argument parsing.</p> <p>The good news however is that we only need to compare two values per field. Consider <code>enum Choice { #[default] A, B, C }</code>. Since we know that at most only one of the two values was provided, since if our idea works, the argument parser rejects if the user provides both.</p> <p>From an external point of view, there are four options <code>A (default), A (provided), B and C</code>. However, we have to decide on a merged value for each pair just using their values <code>A, B and C</code>. If values B or C, on both <code>find xor verify</code>, are given it is simple: by the XOR-or-default condition, B and C are always the user selected values. For A, we luckily do actually not need to care whether it was provided or not. If the other side provides B or C, it was the default (see previous), and if it is A, then we can simply use A.</p> <p>-&gt; I feel the merging/selecting would likely be reasonably possible with this fairly simple scenario in mind, but haven't put much thought into searching counter arguments where it doesn't work (or proving that it always works for that matter).</p> </blockquote> <p>Yet, it would still all be a bit too hairy for my liking considering we use the <code>derive</code> API to get a nice declarative, clean looking argument parser.</p> <p>Positives:</p> <ul> <li>The code (likely) compiles</li> <li>The interface becomes no worse than it is today</li> </ul> <p>Negatives:</p> <ul> <li>Custom merging logic on top of the declarative API</li> </ul> <p>Currently my preferred choice, mostly because it won't worsen the current user experience.</p> <h1 id="idea-2-naming-a-command">Idea 2: naming a command</h1> <p>If we could name one of these derived commands, then we might get away with saying to <code>clap</code>: "the fields within this named command are incompatible with the fields in this other named command"; i.e. I'm trying to apply some constraint.</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span>#[</span><span style="color:#bf616a;">command</span><span>(next_help_heading = &quot;</span><span style="color:#a3be8c;">Find MSRV options</span><span>&quot;)] </span><span style="color:#b48ead;">pub struct </span><span>FindOpts { </span><span> #[</span><span style="color:#bf616a;">arg</span><span>(long, conflicts_with = &quot;</span><span style="color:#a3be8c;">linear</span><span>&quot;)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">bisect</span><span>: </span><span style="color:#b48ead;">bool</span><span>, </span><span> </span><span> #[</span><span style="color:#bf616a;">arg</span><span>(long, conflicts_with = &quot;</span><span style="color:#a3be8c;">bisect</span><span>&quot;)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">linear</span><span>: </span><span style="color:#b48ead;">bool</span><span>, </span><span> </span><span> </span><span style="color:#65737e;">// ... omitted some options for brevity </span><span> </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten, id = &quot;</span><span style="color:#a3be8c;">runner_opts_find</span><span>&quot;, conflicts_with = [&quot;</span><span style="color:#a3be8c;">runner_opts_verify</span><span>&quot;])] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">runner_opts</span><span>: RunnerOptsFind, </span><span>} </span><span> </span><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span>#[</span><span style="color:#bf616a;">command</span><span>(next_help_heading = &quot;</span><span style="color:#a3be8c;">Verify options</span><span>&quot;)] </span><span style="color:#b48ead;">pub struct </span><span>VerifyOpts { </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten, id = &quot;</span><span style="color:#a3be8c;">runner_opts_verify</span><span>&quot;)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">runner_opts</span><span>: RunnerOpts, </span><span> </span><span> #[</span><span style="color:#bf616a;">arg</span><span>(long, value_name = &quot;</span><span style="color:#a3be8c;">rust-version</span><span>&quot;)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">rust_version</span><span>: Option&lt;BareVersion&gt;, </span><span>} </span></code></pre> <p>Unfortunately, this idea doesn't quite cut it. Deriving <code>flatten</code> and supplying an <code>id</code> at the same time is not allowed:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>error: methods are not allowed for flattened entry </span><span> --&gt; src\cli\find_opts.rs:53:15 </span><span> | </span><span>53 | #[command(flatten, id = &quot;runner_opts_find&quot;, conflicts_with = [&quot;runner_opts_verify&quot;])] </span><span> | ^^^^^^^ </span></code></pre> <p>--</p> <p>If this idea would have worked, we also would still need to merge the <code>Opts</code> together, like in the last example.</p> <p>The other reason why this idea is doomed is that when you put the id on the whole group of arguments, I suspect that once you give any argument in either the top level or the subcommand, all other values in the group must also be from the same group; otherwise they would be considered in conflict.</p> <p>Considering <code>cargo msrv {group_top_level} verify {group_subcommand}</code>, then <code>cargo msrv --target x --add-component y verify</code> or <code>cargo msrv verify --target x --add-component y</code> would be fine, but mixing it up like <code>cargo msrv --target x verify --add-component y</code> would be in conflict.</p> <p>Positives:</p> <ul> <li>I can dream about a declarative only solution</li> <li>Maybe this would work on an argument level? <ul> <li>I haven't tried that yet</li> </ul> </li> </ul> <p>Negatives:</p> <ul> <li>Doesn't compile</li> <li>Likely wouldn't work in the first place</li> <li>If it did, it would likely work per whole group, so I would have to use a group per argument</li> </ul> <h1 id="idea-3-separate-structs-groups-and-conflicts">Idea 3: Separate structs, groups and conflicts</h1> <p>I also wanted to try a variation on the previous idea: this idea instead provides two 'separately' nameable things, which then can be marked to be in conflict with one another.</p> <p>We only let the top level variant be in conflict with the subcommand variant, since the latter cannot exist without the former.</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span>#[</span><span style="color:#bf616a;">command</span><span>(next_help_heading = &quot;</span><span style="color:#a3be8c;">Find MSRV options</span><span>&quot;)] </span><span style="color:#b48ead;">pub struct </span><span>FindOpts { </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">runner_opts</span><span>: RunnerOptsFind, </span><span>} </span><span> </span><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span>#[</span><span style="color:#bf616a;">command</span><span>(next_help_heading = &quot;</span><span style="color:#a3be8c;">Verify options</span><span>&quot;)] </span><span style="color:#b48ead;">pub struct </span><span>VerifyOpts { </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">runner_opts</span><span>: RunnerOptsVerify, </span><span> </span><span> </span><span style="color:#65737e;">// ... etc. </span><span>} </span><span> </span><span style="color:#65737e;">// NB: Should be kept in sync with `RunnerOptsVerify`! </span><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span>#[</span><span style="color:#bf616a;">group</span><span>(id = &quot;</span><span style="color:#a3be8c;">runner_opts_find</span><span>&quot;, conflicts_with = &quot;</span><span style="color:#a3be8c;">runner_opts_verify</span><span>&quot;, multiple = true)] </span><span style="color:#b48ead;">pub struct </span><span>RunnerOptsFind { </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">rust_releases_opts</span><span>: RustReleasesOpts, </span><span> </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">toolchain_opts</span><span>: ToolchainOpts, </span><span> </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">cargo_check_opts</span><span>: CheckCommandOpts, </span><span>} </span><span> </span><span style="color:#65737e;">// NB: Should be kept in sync with `RunnerOptsFind`! </span><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span>#[</span><span style="color:#bf616a;">group</span><span>(id = &quot;</span><span style="color:#a3be8c;">runner_opts_verify</span><span>&quot;, multiple = true)] </span><span style="color:#b48ead;">pub struct </span><span>RunnerOptsVerify { </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">rust_releases_opts</span><span>: RustReleasesOpts, </span><span> </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">toolchain_opts</span><span>: ToolchainOpts, </span><span> </span><span> #[</span><span style="color:#bf616a;">command</span><span>(flatten)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">cargo_check_opts</span><span>: CheckCommandOpts, </span><span>} </span></code></pre> <p>However, running results in:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cargo run -- msrv verify --target x </span><span> Compiling cargo-msrv v0.16.0-beta.22 (C:\ws\cargo-msrv) </span><span> Finished `dev` profile [unoptimized + debuginfo] target(s) in 11.24s </span><span> Running `target\debug\cargo-msrv.exe msrv verify --target x` </span><span>thread &#39;main&#39; panicked at C:\Users\x\.cargo\registry\src\index.crates.io-6f17d22bba15001f\clap_builder-4.5.7\src\builder\debug_asserts.rs:314:13: </span><span>Command msrv: Argument group &#39;runner_opts_find&#39; conflicts with non-existent &#39;runner_opts_verify&#39; id </span><span>stack backtrace: </span></code></pre> <p>There's probably a logical explanation here why this doesn't work; I guess I don't properly understand how <code>ArgGroup</code> works...</p> <p>Neutral:</p> <ul> <li>I don't seems to understand <code>ArgGroup</code> properly yet</li> </ul> <h1 id="idea-4-global-true">Idea 4: global = true</h1> <p>This was the first idea I had when I read the originally reported issue. It will again not satisfy goal (2), because marking a command as global, makes it, well, available to all subcommands.</p> <p>This could however be a pragmatic choice, since in the current form they're already present via the top level <code>FindOpts</code>.</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">derive</span><span>(Debug, Args)] </span><span>#[</span><span style="color:#bf616a;">command</span><span>(next_help_heading = &quot;</span><span style="color:#a3be8c;">Toolchain options</span><span>&quot;)] </span><span style="color:#b48ead;">pub struct </span><span>ToolchainOpts { </span><span> #[</span><span style="color:#bf616a;">arg</span><span>(long, value_name = &quot;</span><span style="color:#a3be8c;">TARGET</span><span>&quot;, global = true)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">target</span><span>: Option&lt;String&gt;, </span><span> </span><span> #[</span><span style="color:#bf616a;">arg</span><span>(long, value_name = &quot;</span><span style="color:#a3be8c;">COMPONENT</span><span>&quot;, global = true)] </span><span> </span><span style="color:#b48ead;">pub </span><span style="color:#bf616a;">add_component</span><span>: Vec&lt;String&gt;, </span><span>} </span></code></pre> <p>When I looked at this idea in more detail, I actually found that this may be considered slightly worse since <code>cargo msrv list --target x</code> would be valid when <code>global = true</code>, while the current form 'only' allows <code>cargo msrv --target x list</code> (the residue of having the <code>FindOpts</code> at the top level). Plus, the current form does, for better or worse, not show the <code>FindOpts</code> help text when providing <code>--help</code> on the subcommand: i.e. <code>cargo msrv verify --help</code>.</p> <p>Now, it also doesn't really satisfy goal (1). For example: <code>cargo msrv --add-component a --add-component b --target x verify --add-component c --add-component d --target y</code> , produces:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>toolchain: ToolchainContext { </span><span> target: &quot;y&quot;, </span><span> components: [ </span><span> &quot;c&quot;, </span><span> &quot;d&quot;, </span><span> ], </span><span> }, </span></code></pre> <p>While, <code>cargo msrv --target x --add-component a verify --add-component b --target y --target z</code> , does result in :</p> <p><code>error: the argument '--target &lt;TARGET&gt;' cannot be used multiple times</code></p> <p>Positives:</p> <ul> <li>The code compiles</li> <li>Most simple scenarios work</li> <li>It doesn't need hairy merging code (on the surface)</li> </ul> <p>Negatives:</p> <ul> <li>The interface will allow additional ignored arguments for subcommands other than verify <ul> <li>This list can grow large if all of <code>RunnerOpts</code> arguments are included</li> </ul> </li> <li>Mixed usage does not limit the <code>num_args</code> for <code>Args</code> on a global level</li> <li>Mixed usage overwrites values specified on a deeper subcommand level <ul> <li>Behaviour can possibly be changed with <code>ArgAction</code> (?)</li> <li>This wouldn't be a problem if on a global level it is possible to enforce that arguments could be only provided to one level / for non collection types, that the <code>num_args</code> would be complied with on the global level</li> </ul> </li> <li>Mixed usage is unintuitive for a user</li> </ul> <h1 id="remarks-on-using-clap">Remarks on using Clap</h1> <p><em>In this likely non-standard use case.</em></p> <h2 id="clap-s-api-surface">Clap's API surface</h2> <p><code>Clap</code> has grown quite the set of options. There's something for everyone. It may very well still be possible to fulfill the use case I described above. Just because I couldn't find it after searching for it for a while doesn't mean it doesn't exist. In the end, I decided to choose pragmatic over pure.</p> <h2 id="skip">Skip</h2> <p>Many attributes define the magic <code>skip</code> option. It is available for <code>ArgGroup</code> and <code>Arg</code>. It is not available to <code>Command</code> in general (a variation with different behaviour is on <code>Subcommand</code>).</p> <p>At first it sounded like logical name for the something I'm looking for, however it ignores fields and sets a provided expression (or Default::default if omitted), so this doesn't feel like it works.</p> <p>Aside: what kind of expression should I even put in, even if it fulfilled my use case; you would need to be able to refer to something which ought to be skipped. In my case that would be conditionally skipping if the subcommand is <code>Some(s) where S is not Verify</code></p> <h2 id="command-args-conflicts-with-subcommands">Command::args_conflicts_with_subcommands</h2> <p>For a second, I hoped that if a command could be marked with <code>args_conflicts_with_subcommands</code> we could then reject all<br /> subcommands which are not <code>verify</code>, but this method <code>args_conflicts_with_subcommands(...)</code> takes a bool; not a list of subcommands... 😭, and has another use case altogether...</p> <h2 id="shared-value-state">Shared value state</h2> <p>At some point I had a fleeting wish that there could be a <code>GroupArg</code> (or something else altogether) where values share their state.</p> <p>I.e. given a group <code>G</code>, defined on the top level as <code>G_t</code> and on a subcommand as <code>G_s</code>, when setting a value for <code>G_t</code> or <code>G_s</code>, their value state would be shared. Still just this would not solve the most prominent problem, i.e. for the user we want to explicitly allow just one of each argument for any <code>i</code> in <code>G_i</code> to be defined...</p> <p>I suppose, this is not dissimilar to <code>global = true</code> on <code>Args</code>, except for some action override since <code>global = true</code> doesn't work intuitively by default for args defined on mixed levels.</p> <h2 id="derive-and-groups">Derive and groups</h2> <p>When using derive, it is not always straightforward to get an <code>Arg</code> from a group, since we have the derived structs; not <code>ArgMatches</code>. With <code>ArgMatches</code> you can, if I understand the documentation correctly, get the value of the group using one of the <code>get_{}</code> methods on <code>ArgMatches</code>.</p> <p>Possibly I could hack this together by using <code>ArgAction::set</code> with <code>Command::args_override_self(true)</code> ..?</p> <p>I still think I don't really understand the true value of <code>ArgGroup</code>. The "a group of shared commands" I was looking for is probably something else entirely. 😉</p> <h1 id="conclusion">Conclusion</h1> <p>If you, dear reader, by any chance know how to solve the issue of this post, or have other ideas: I love <a href="https://github.com/foresterre/cargo-msrv/issues/936">to learn</a> (about them). <sup>You also send me a mail, or contact me in another way, see my GitHub profile.</sup></p> <p>In the mean time, I suppose that this just isn't the way forward. To meet goal (2) in particular, having some shared arguments also at the top level makes solving the issue a lot harder. I tried to add constraints on how this definition could be used, but none of these really worked out.</p> <p>Maybe it is finally time to sunset the <code>cargo msrv</code> top level command and introduce a dedicated subcommand for "find your msrv" instead.</p> <p>p.s. I kind of wrote this all on a whim, so it may be full of mistakes 🫧.</p> <h1 id="thanks">Thanks!</h1> <p>This post is based on an <a href="https://github.com/foresterre/cargo-msrv/issues/936">issue</a> originally reported by <a href="https://github.com/Finomnis">Finomnis</a> , thanks!</p> A tale of setInterval and useEffect in React Native 2024-04-19T00:00:00+00:00 2024-04-19T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/setinterval-and-friends-in-react-native/ <p>At work, I've been building a new <a href="https://reactnative.dev">React Native</a> app. Initially, I wanted this app to be as simple as possible, to allow for quick iterative cycles.</p> <p>Now this app needs to refetch a certain resource from the server in regular intervals. I have to admit: I briefly considered to use <a href="https://tanstack.com/query/v3/docs/framework/react/react-native">React Query</a>, but decided it wasn't quite time for that yet. Simple first. Complex later.</p> <p>I looked at the React Native docs in an attempt for figure out what the canonical way of doing this was. The docs told me: "hey, you can use the <code>Timers</code> module which contains <code>setInterval</code> and <code>clearInterval</code>". Just what I needed!</p> <p><code>setInterval</code> executes a callback after a given milliseconds delay. It doesn't have the option to immediately fire, however. Luckily it's not hard to work around this: for example by just executing the function provided in the callback first.</p> <p>As an alternative, <a href="https://developer.mozilla.org/en-US/docs/Web/API/setInterval#ensure_that_execution_duration_is_shorter_than_interval_frequency">MDN</a> suggested that you could also use <code>setTimeout</code>, although in my case, the execution duration is shorter than the interval frequency, so I figured everything should be fine 🤞.</p> <p>To give an idea of what the code looked like:</p> <pre data-lang="typescript" style="background-color:#2b303b;color:#c0c5ce;" class="language-typescript "><code class="language-typescript" data-lang="typescript"><span style="color:#b48ead;">export default function </span><span style="color:#8fa1b3;">MyComponent</span><span>(): React.JSX.Element { </span><span> </span><span style="color:#8fa1b3;">useEffect</span><span>(() </span><span style="color:#b48ead;">=&gt; </span><span>{ </span><span> </span><span style="color:#96b5b4;">setInterval</span><span>(() </span><span style="color:#b48ead;">=&gt; </span><span>void </span><span style="color:#8fa1b3;">fetchResource</span><span>()); </span><span> }, []); </span><span>} </span><span> </span><span style="color:#b48ead;">async function </span><span style="color:#8fa1b3;">fetchResource</span><span>() { </span><span> </span><span style="color:#b48ead;">try </span><span>{ </span><span> </span><span style="color:#b48ead;">const </span><span style="color:#bf616a;">result </span><span>= </span><span style="color:#b48ead;">await </span><span style="color:#bf616a;">client</span><span>.</span><span style="color:#bf616a;">resource</span><span>.</span><span style="color:#8fa1b3;">fetch</span><span>(); </span><span> </span><span style="color:#8fa1b3;">setOk</span><span>(</span><span style="color:#bf616a;">result</span><span>); </span><span> } </span><span style="color:#b48ead;">catch </span><span>(</span><span style="color:#bf616a;">error</span><span>: unknown) { </span><span> </span><span style="color:#b48ead;">const </span><span style="color:#bf616a;">error </span><span>= </span><span style="color:#bf616a;">ResourceErrorParser</span><span>.</span><span style="color:#8fa1b3;">parseError</span><span>(</span><span style="color:#bf616a;">error</span><span>); </span><span> </span><span style="color:#8fa1b3;">setError</span><span>(</span><span style="color:#bf616a;">error</span><span>); </span><span> } </span><span>} </span></code></pre> <p>Now one of the most useful features of React Native, is its ability to live inspect changes you just made. Running the <a href="https://metrobundler.dev">Metro</a> development server in combination with a debug build of the app gives you live updates out of the box. For me, that's simply running <code>npm run start</code> to start Metro and in a second terminal tab <code>npm run android</code> to build a debug build of the app and install it on a device (or emulator).</p> <p>Now, changes made to components or other code can be updated, and the changes can be observed on the app without rebuilding. Great!</p> <p>One day, I was checking the logs in the Metro terminal app, and I there seemed to be a few too many fetches to the backend. Normally, it should refetch the resource every 10 seconds or so (or immediately after certain actions), now it was making requests to fetch the resource tens of times per second. Whoops.</p> <p>So what happened? The callback in <code>useEffect</code> was being rerendered after each change in the code. And as a result, the <code>setInterval</code> function was being rerun as well. On repeat. Oops 😅.</p> <p>Luckily, it can be fixed! As it turns out, <code>setInterval</code> can return a clean up function, which runs when the component unmounts (or when the props get updated and the dependencies provided to the <code>useEffect</code> have been changed, or even every rerender if no dependency array is provided):</p> <pre data-lang="typescript" style="background-color:#2b303b;color:#c0c5ce;" class="language-typescript "><code class="language-typescript" data-lang="typescript"><span style="color:#b48ead;">type </span><span>Id = ReturnType&lt;typeof setInterval&gt;; </span><span> </span><span style="color:#65737e;">// elsewhere: </span><span style="color:#b48ead;">export default function </span><span style="color:#8fa1b3;">OtherComponent</span><span>(): React.JSX.Element { </span><span> </span><span style="color:#b48ead;">const </span><span>[</span><span style="color:#bf616a;">refetchId</span><span>, </span><span style="color:#bf616a;">setRefetchId</span><span>] = </span><span style="color:#8fa1b3;">useState</span><span>&lt;Id | undefined&gt;(</span><span style="color:#d08770;">undefined</span><span>); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span>&lt;MyComponent id={</span><span style="color:#bf616a;">refreshId</span><span>} setId={</span><span style="color:#bf616a;">setRefetchId</span><span>} onClear={() =&gt; </span><span style="color:#8fa1b3;">setRefetchId</span><span>(</span><span style="color:#bf616a;">undefined</span><span>)} /&gt;; </span><span>} </span><span> </span><span> </span><span style="color:#b48ead;">export default function </span><span style="color:#8fa1b3;">MyComponent</span><span>(</span><span style="color:#bf616a;">props</span><span>: { </span><span style="color:#bf616a;">id</span><span>?: Id; </span><span style="color:#8fa1b3;">setId</span><span>: (</span><span style="color:#bf616a;">id</span><span>: Id) </span><span style="color:#b48ead;">=&gt; </span><span>void; </span><span style="color:#8fa1b3;">onClear</span><span>: () </span><span style="color:#b48ead;">=&gt; </span><span>void; }): React.JSX.Element { </span><span> </span><span style="color:#8fa1b3;">useEffect</span><span>(() </span><span style="color:#b48ead;">=&gt; </span><span>{ </span><span> </span><span style="color:#b48ead;">const </span><span style="color:#bf616a;">id </span><span>= </span><span style="color:#96b5b4;">setInterval</span><span>(() </span><span style="color:#b48ead;">=&gt; </span><span>void </span><span style="color:#8fa1b3;">fetchResource</span><span>()); </span><span> </span><span style="color:#bf616a;">props</span><span>.</span><span style="color:#8fa1b3;">setId</span><span>(</span><span style="color:#bf616a;">id</span><span>); </span><span> </span><span> </span><span style="color:#b48ead;">return </span><span>() </span><span style="color:#b48ead;">=&gt; </span><span>({ </span><span> </span><span style="color:#8fa1b3;">if </span><span>(</span><span style="color:#bf616a;">id</span><span>) { </span><span> </span><span style="color:#96b5b4;">clearInterval</span><span>(</span><span style="color:#bf616a;">id</span><span>); </span><span> } </span><span> }); </span><span> }, []); </span><span>} </span></code></pre> <p>Note that the dependency array can be empty, since we only want to disable the fetch task if the component is unmounted. Otherwise it can just do its thing and fetch the resource at each specific interval. And you know what. That's good enough for me, ... at least for now.</p> <p><em>Wishlist: I wish there was a way to see all active useIntervals and useTimeouts: if you know a way (without storing the id with some state management utility): please <a href="https://gribnau.dev/posts/setinterval-and-friends-in-react-native/(https://github.com/foresterre/foresterre.github.io/discussions)">let me know</a> 🙏</em></p> <p><em>I did end up extending the functionality ever so slightly: I added the option to toggle refreshing altogether and changed when refetches happen, to reschedule the interval when a manual refresh happened (which happens after certain actions) 😅.</em></p> <h1 id="feedback-discussion">Feedback &amp; discussion</h1> <p>Feedback is most welcome. Feel free to discuss at <a href="https://github.com/foresterre/foresterre.github.io/discussions">GitHub</a>.</p> What's new in Yare 3.0.0 (A lean parameterized testing macro for Rust) 2024-03-08T00:00:00+00:00 2024-03-08T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/parameterized-macro-yare-v3/ <h1 id="what-is-yare">What is Yare?</h1> <p><a href="https://github.com/foresterre/yare">Yare</a> is a lean parameterized testing macro for Rust.</p> <p>This practically means that when using <code>#[yare::parameterized]</code>, it is easier to write a test scenario, which can be tested against multiple different inputs. Each set of inputs is a separate test case.</p> <p>For example:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>yare::parameterized; </span><span> </span><span>#[</span><span style="color:#bf616a;">parameterized</span><span>( </span><span> apple = { Fruit::Apple, &quot;</span><span style="color:#a3be8c;">apple</span><span>&quot; }, </span><span> pear = { Fruit::Pear, &quot;</span><span style="color:#a3be8c;">pear</span><span>&quot; }, </span><span> blackberry = { Fruit::</span><span style="color:#bf616a;">Bramble</span><span>(BrambleFruit::Blackberry), &quot;</span><span style="color:#a3be8c;">blackberry</span><span>&quot; }, </span><span>)] </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">a_fruity_test</span><span>(</span><span style="color:#bf616a;">fruit</span><span>: Fruit, </span><span style="color:#bf616a;">name</span><span>: &amp;</span><span style="color:#b48ead;">str</span><span>) { </span><span> assert_eq!(fruit.</span><span style="color:#96b5b4;">name_of</span><span>(), name) </span><span>} </span></code></pre> <p>The above scenario will generate 3 test cases: <code>apple</code>, <code>pear</code> and <code>blackberry</code>, while it was only necessary to specify the scenario once.</p> <p>As you might imagine, if your add more tests, the removal of duplicated test cases does not only save writing (and maintenance!) time, but makes it also easier to keep scenarios the same for a large set of inputs (especially while refactoring code, changes to test scenarios may sneak in, which should also have applied to equivalent inputs).</p> <h1 id="what-s-new-in-3-0-0">What's new in 3.0.0</h1> <h2 id="custom-test-macro-e-g-tokio-test">Custom test macro (e.g. tokio::test)</h2> <p>Prior to 3.0.0, Yare would always generate test cases with the Rust built in <code>#[test]</code> attribute. While it is exceptionally useful to have this macro built in, at times you may want to use a different macro because the built-in one doesn't support a feature you need.</p> <p>A common example is the <code>tokio::test</code> macro, when using the <a href="https://github.com/tokio-rs/tokio">tokio</a> asynchronous runtime. While you could create your own <a href="https://docs.rs/tokio/latest/tokio/runtime/struct.Runtime.html#method.spawn">Runtime</a> and spawn futures onto this runtime for your test cases, it is perhaps not as elegant as using the <a href="https://docs.rs/tokio/latest/tokio/attr.test.html">tokio::test</a> macro.</p> <p>With this use case in mind, Yare can now be used with user specified test macro's. If none is specified, the Rust built-in <code>#[test]</code> will be used.</p> <p><strong>Example</strong></p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>yare::parameterized; </span><span> </span><span>#[</span><span style="color:#bf616a;">parameterized</span><span>( </span><span> zero_wait = { 0, 0 }, </span><span> show_paused = { 500, 0 }, </span><span>)] </span><span>#[</span><span style="color:#bf616a;">test_macro</span><span>(tokio::</span><span style="color:#bf616a;">test</span><span>(start_paused = true))] </span><span>async </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">test</span><span>(</span><span style="color:#bf616a;">wait</span><span>: </span><span style="color:#b48ead;">u64</span><span>, </span><span style="color:#bf616a;">time_elapsed</span><span>: </span><span style="color:#b48ead;">u128</span><span>) { </span><span> </span><span style="color:#b48ead;">let</span><span> start = std::time::Instant::now(); </span><span> tokio::time::sleep(tokio::time::Duration::from_millis(wait)).await; </span><span> </span><span> assert_eq!(time_elapsed, start.</span><span style="color:#96b5b4;">elapsed</span><span>().</span><span style="color:#96b5b4;">as_millis</span><span>()); </span><span>} </span><span> </span><span style="color:#65737e;">// to use `start_paused = true`, enable the test-util feature for your tokio dependency </span><span style="color:#65737e;">// example inspired by: https://tokio.rs/tokio/topics/testing </span></code></pre> <h3 id="how-does-it-work">How does it work?</h3> <p>To make this work, the <code>#[parameterized(...)]</code> attribute in Yare parses all attributes placed after it (all attributes must be placed on top of the test function):</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>syn::parse::{Parse, ParseStream, Result}; </span><span> </span><span style="color:#b48ead;">enum </span><span>Attribute { </span><span> </span><span style="color:#65737e;">/// A regular attribute, which isn&#39;t named &quot;test_macro&quot; </span><span> </span><span style="color:#65737e;">/// NB: Attribute and syn::Attribute are not the same! </span><span> Normal(syn::Attribute), </span><span> </span><span style="color:#65737e;">// An attribute named &quot;test_macro&quot; </span><span> TestMacro(syn::Meta), </span><span>} </span><span> </span><span style="color:#b48ead;">pub struct </span><span>TestFn { </span><span> </span><span style="color:#bf616a;">attributes</span><span>: Vec&lt;Attribute&gt;, </span><span> </span><span style="color:#bf616a;">fun</span><span>: syn::ItemFn, </span><span>} </span><span> </span><span style="color:#b48ead;">impl </span><span>Parse </span><span style="color:#b48ead;">for </span><span>TestFn { </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">parse</span><span>(</span><span style="color:#bf616a;">input</span><span>: ParseStream) -&gt; Result&lt;</span><span style="color:#b48ead;">Self</span><span>&gt; { </span><span> Ok(TestFn { </span><span> attributes: input </span><span> .</span><span style="color:#96b5b4;">call</span><span>(syn::Attribute::parse_outer)? </span><span> .</span><span style="color:#96b5b4;">into_iter</span><span>() </span><span> .</span><span style="color:#96b5b4;">map</span><span>(|</span><span style="color:#bf616a;">attr</span><span>| { </span><span> </span><span style="color:#b48ead;">if</span><span> attr.</span><span style="color:#96b5b4;">path</span><span>().</span><span style="color:#96b5b4;">is_ident</span><span>(&quot;</span><span style="color:#a3be8c;">test_macro</span><span>&quot;) { </span><span> attr.parse_args::&lt;syn::Meta&gt;().</span><span style="color:#96b5b4;">map</span><span>(Attribute::TestMacro) </span><span> } </span><span style="color:#b48ead;">else </span><span>{ </span><span> Ok(Attribute::Normal(attr)) </span><span> } </span><span> }) </span><span> .collect::&lt;Result&lt;Vec&lt;_&gt;&gt;&gt;()?, </span><span> fun: input.</span><span style="color:#96b5b4;">parse</span><span>()?, </span><span> }) </span><span> } </span><span>} </span></code></pre> <p>So for the following Rust source code:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">parameterized</span><span>( </span><span> red = { FunctionColor::Red }, </span><span> blue = { FunctionColor::Blue }, </span><span>)] </span><span>#[</span><span style="color:#bf616a;">test_macro</span><span>(function_color::test_macro)] </span><span>#[</span><span style="color:#bf616a;">should_panic</span><span>] </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">test</span><span>(</span><span style="color:#bf616a;">color</span><span>: FunctionColor) { </span><span> </span><span style="color:#65737e;">// ... </span><span>} </span></code></pre> <p>We end up with 1 test_macro attribute and 1 "normal" attribute (i.e. not test_macro):</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>vec![ </span><span> Attribute::TestMacro(syn::Meta::parse(&quot;</span><span style="color:#a3be8c;">test_macro(function_color::test_macro)</span><span>&quot;)), </span><span style="color:#65737e;">// hypothetically, if a &amp;str would be a syn::ParseStream </span><span> Attribute::Normal(syn::Attribute::parse(&quot;</span><span style="color:#a3be8c;">#[should_panic]</span><span>&quot;)), </span><span>]; </span></code></pre> <p>In the code generation phase, we can obtain these separately from <code>TestFn</code>:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">impl </span><span>TestFn { </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">attributes</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>) -&gt; Vec&lt;syn::Attribute&gt; { </span><span> </span><span style="color:#bf616a;">self </span><span> .attributes </span><span> .</span><span style="color:#96b5b4;">iter</span><span>() </span><span> .</span><span style="color:#96b5b4;">filter_map</span><span>(Attribute::to_normal) </span><span> .</span><span style="color:#96b5b4;">collect</span><span>() </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">test_macro_attribute</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>) -&gt; syn::Meta { </span><span> </span><span style="color:#bf616a;">self</span><span>.attributes </span><span> .</span><span style="color:#96b5b4;">iter</span><span>() </span><span> .</span><span style="color:#96b5b4;">find_map</span><span>(Attribute::to_test_macro) </span><span style="color:#65737e;">// NB: We elsewhere asserted that there&#39;s at most one of these. </span><span> .</span><span style="color:#96b5b4;">unwrap_or_else</span><span>(|| { </span><span> </span><span style="color:#65737e;">// A definition for the default #[test] macro </span><span> syn::Meta::Path(syn::Path::from(syn::Ident::new( </span><span> &quot;</span><span style="color:#a3be8c;">test</span><span>&quot;, </span><span> proc_macro2::Span::call_site(), </span><span> ))) </span><span> }) </span><span> } </span><span>} </span></code></pre> <p>And finally during generation itself:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">impl </span><span>TestCase { </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">to_token_stream</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>, </span><span style="color:#bf616a;">test_fn</span><span>: &amp;TestFn) -&gt; Result&lt;proc_macro2::TokenStream&gt; { </span><span> </span><span style="color:#b48ead;">let</span><span> test_meta = test_fn.</span><span style="color:#96b5b4;">test_macro_attribute</span><span>(); </span><span> </span><span style="color:#b48ead;">let</span><span> attributes = test_fn.</span><span style="color:#96b5b4;">attributes</span><span>(); </span><span> </span><span style="color:#65737e;">// Many other things omitted... </span><span> </span><span> Ok(quote::quote! { </span><span> #[#</span><span style="color:#bf616a;">test_meta</span><span>] </span><span style="color:#65737e;">// &lt;-- Our custom macro attribute, or #[test] if none was specified </span><span> #(#attributes)* </span><span style="color:#65737e;">// &lt;-- The &quot;normal&quot; attributes, reproduced </span><span> #visibility #constness #asyncness #unsafety #abi </span><span style="color:#b48ead;">fn </span><span>#</span><span style="color:#96b5b4;">identifier</span><span>() #return_type { </span><span> #bindings </span><span> #body </span><span> } </span><span> }) </span><span> } </span><span>} </span></code></pre> <p>When the code generation phase of the macro has been completed, the parameterized test function will have been substituted by two separate test functions:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">function_color</span><span>::</span><span style="color:#bf616a;">test_macro</span><span>] </span><span>#[</span><span style="color:#bf616a;">should_panic</span><span>] </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">red</span><span>() { </span><span> </span><span style="color:#65737e;">// ... </span><span>} </span><span> </span><span>#[</span><span style="color:#bf616a;">function_color</span><span>::</span><span style="color:#bf616a;">test_macro</span><span>] </span><span>#[</span><span style="color:#bf616a;">should_panic</span><span>] </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">blue</span><span>() { </span><span> </span><span style="color:#65737e;">// ... </span><span>} </span></code></pre> <h3 id="gotchas-to-be-aware-of">Gotchas to be aware of</h3> <p><strong>The <code>#[test_macro(...)]</code> attribute must be placed after <code>#[parameterized]</code></strong></p> <p>Like all macro's, <code>yare::parameterized</code> can only parse the available scope. Since <code>yare::parameterized</code> is supposed to be placed on top of functions, we can access our own attribute (which we use to parse the test case identifier and arguments for test cases) and the function underneath (used to specify the parameters and test scenario in the function body).</p> <p>While <code>yare::parameterized</code> does have access to attributes placed after it, the ones which come before it, are inaccessible.</p> <p>Subsequently, <code>yare::parameterized</code> can only recognize placements of <code>#[test_macro(...)]</code> which come after it.</p> <p>So, the following is ok:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>yare::parameterized; </span><span> </span><span>#[</span><span style="color:#bf616a;">parameterized</span><span>( </span><span> wow = { &quot;</span><span style="color:#a3be8c;">wow!</span><span>&quot; }, </span><span> whew = { &quot;</span><span style="color:#a3be8c;">whew!</span><span>&quot; }, </span><span>)] </span><span>#[</span><span style="color:#bf616a;">test_macro</span><span>(tokio::test)] </span><span>async </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">test</span><span>(</span><span style="color:#bf616a;">sample</span><span>: &amp;</span><span style="color:#b48ead;">str</span><span>) { </span><span> </span><span style="color:#65737e;">// ... </span><span>} </span></code></pre> <p>While this doesn't work:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>yare::parameterized; </span><span> </span><span>#[</span><span style="color:#bf616a;">test_macro</span><span>(tokio::test)] </span><span>#[</span><span style="color:#bf616a;">parameterized</span><span>( </span><span> wow = { &quot;</span><span style="color:#a3be8c;">wow!</span><span>&quot; }, </span><span> whew = { &quot;</span><span style="color:#a3be8c;">whew!</span><span>&quot; }, </span><span>)] </span><span>async </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">test</span><span>(</span><span style="color:#bf616a;">sample</span><span>: &amp;</span><span style="color:#b48ead;">str</span><span>) { </span><span> </span><span style="color:#65737e;">// ... </span><span>} </span></code></pre> <p><strong>One <code>#[test_macro(...)]</code> per parameterized test function</strong></p> <p>Yare currently accepts one <code>#[test_macro(...)]</code> for a parameterized test function. The following is not allowed and will return an error:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>yare::parameterized; </span><span> </span><span>#[</span><span style="color:#bf616a;">parameterized</span><span>( </span><span> wow = { &quot;</span><span style="color:#a3be8c;">wow!</span><span>&quot; }, </span><span> whew = { &quot;</span><span style="color:#a3be8c;">whew!</span><span>&quot; }, </span><span>)] </span><span>#[</span><span style="color:#bf616a;">test_macro</span><span>(tokio::test)] </span><span>#[</span><span style="color:#bf616a;">test_macro</span><span>(tokio::</span><span style="color:#bf616a;">test</span><span>(start_paused = true))] </span><span>async </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">test</span><span>(</span><span style="color:#bf616a;">sample</span><span>: &amp;</span><span style="color:#b48ead;">str</span><span>) { </span><span> </span><span style="color:#65737e;">// ... </span><span>} </span></code></pre> <p>Returned error:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>error: Expected at most 1 #[test_macro(...)] attribute, but 2 were given </span><span>--&gt; tests/fail/multiple_test_macro_attributes.rs:8:14 </span><span> | </span><span>8 | #[test_macro(tokio::test(start_paused = true))] </span><span> | ^^^^^ </span></code></pre> <p>The reason is that it's unclear what should happen when multiple <code>#[test_macro(...)]</code> attributes are present. Should the first one be used by <code>#[parameterized(...)]</code>, and subsequent be left in place? Or would that just be confusing, if only because the test author will need to keep track of which macro uses which attributes.</p> <p><strong>Renaming attributes</strong></p> <p>Yare's parameterized attribute can be used with a different name if you would like, by rebinding the target import part under a local name:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>yare::parameterized as test_macro_for_twitchcraft_and_lizardry; </span><span style="color:#65737e;">// &lt;-- Rebinding! </span><span> </span><span>#[</span><span style="color:#bf616a;">test_macro_for_twitchcraft_and_lizardry</span><span>( &lt;-- Usage </span><span> gryffinroar = { &quot;</span><span style="color:#a3be8c;">Gryffinroar</span><span>&quot; }, </span><span> hufflefluff = { &quot;</span><span style="color:#a3be8c;">Hufflefluff</span><span>&quot; }, </span><span> ravenpaw = { &quot;</span><span style="color:#a3be8c;">Ravenpaw</span><span>&quot; }, </span><span> slytherfin = { &quot;</span><span style="color:#a3be8c;">Slytherfin</span><span>&quot; }, </span><span>)] </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">houses</span><span>(</span><span style="color:#bf616a;">name</span><span>: &amp;</span><span style="color:#b48ead;">str</span><span>) { </span><span> </span><span style="color:#65737e;">// ... </span><span>} </span></code></pre> <p>However, since the <code>#[test_macro(...)]</code> is parsed by <code>yare::parameterized</code>, it cannot be renamed.</p> <h2 id="extended-function-qualifier-support">Extended function qualifier support</h2> <p>Previously, when Yare was still written to support mostly just the built-in <code>#[test]</code> macro, it wasn't so useful to support the function qualifiers: <code>const</code>, <code>async</code>, <code>unsafe</code> and <code>extern</code>. Firstly, because half of these aren't even supported by <code>#[test]</code>. For the ones that are, namely <code>const</code> and <code>extern</code>, the I deemed the usefulness to be limited. For example: with <code>const</code> you can't use the commonly used <code>assert_eq!</code> since the <code>PartialEq</code> trait is not marked as <code>const</code>. And regarding <code>extern</code>, I've never seen anyone call unit test functions over FFI (but if you do, I would like to know, it does sound fun 😅).</p> <p>However, with custom test macro's, you want at least support for <code>async</code> and maybe for <code>unsafe</code>. Adding the other two is hardly more work, so I also added <code>const</code> and <code>extern</code> for completeness.</p> <p>NB: when specifying one ore more qualifiers in the function definition of your test function, the underlying test macro (whether <code>#[test]</code> or a custom macro <code>#[test_macro(x)]</code>) must also support the specified qualifiers.</p> <p><strong>Example</strong></p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>yare::parameterized; </span><span> </span><span style="color:#65737e;">// NB: The underlying test macro also must support these qualifiers. For example, the default `#[test]` doesn&#39;t support async and unsafe. </span><span> </span><span>#[</span><span style="color:#bf616a;">parameterized</span><span>( </span><span> purple = { &amp;[128, 0, 128] }, </span><span> orange = { &amp;[255, 127, 0] }, </span><span>)] </span><span style="color:#b48ead;">const extern </span><span>&quot;</span><span style="color:#a3be8c;">C</span><span>&quot; </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">has_reds</span><span>(</span><span style="color:#bf616a;">streamed_color</span><span>: &amp;[</span><span style="color:#b48ead;">u8</span><span>]) { </span><span> assert!(streamed_color.</span><span style="color:#96b5b4;">first</span><span>().</span><span style="color:#96b5b4;">is_some</span><span>()); </span><span>} </span></code></pre> <h1 id="ideas-feedback-and-bug-reports-for-yare">Ideas, feedback and bug reports for Yare</h1> <p>Ideas, feedback and bug reports are most welcome. Feel free to open an issue on <a href="https://github.com/foresterre/yare/issues">GitHub</a>.</p> <h1 id="discuss-this-post">Discuss this post</h1> <p>Discuss on <a href="https://reddit.com/r/rust/comments/1b9ijs6/yare_v300_a_lean_parameterized_testing_macro_for/">Reddit</a>.</p> Using the todo! macro to prototype your API in Rust 2023-04-24T00:00:00+00:00 2023-04-24T00:00:00+00:00 Martijn Gribnau https://gribnau.dev/posts/todo-macro-rust/ <p>Let's sketch a situation. You're designing and implementing a library in Rust, for some great idea you had. And, you aim to create a seamless API that makes this library user-friendly not only for others, but also for yourself.</p> <p>One way to figure out the design of the library is to write a prototype, and write the bare minimum code to stub out the the initial API. I hear you think: Rust is not to most convenient prototyping language, because it's quite strict and verbose: we have to satisfy the borrow checker and, in many places, Rust requires you to explicitely type your code. And altough I believe both will help you design better code, I can also understand the argument that it reduces the prototyping velocity at least a bit.</p> <p>Luckily for us, the Rust standard library has a useful tool in its toolbox: the <a href="https://doc.rust-lang.org/std/macro.todo.html">todo!</a> macro.</p> <p>Let's look at an example. Imagine<sup><a href="#footnote1">1</a></sup> we're re-designing a Rust API to fetch Rust releases metadata.</p> <p>We will first prototype a few data structures around the concept of "releases":</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#65737e;">/// A data structure consisting of the set of known Rust releases. </span><span style="color:#65737e;">/// </span><span style="color:#65737e;">/// Whether a release is known, and how much information is known </span><span style="color:#65737e;">/// about a release, depends on the source used to build up this </span><span style="color:#65737e;">/// information. </span><span style="color:#b48ead;">struct </span><span>RustReleases { </span><span> </span><span style="color:#65737e;">// We divide all releases by platform, so we end up with the </span><span> </span><span style="color:#65737e;">// set of available toolchains for each platform. </span><span> </span><span style="color:#bf616a;">registry</span><span>: HashMap&lt;rust_toolchain::Platform, ReleaseRecords&gt;, </span><span>} </span><span> </span><span style="color:#65737e;">/// A set of releases, for a single platform. </span><span style="color:#b48ead;">struct </span><span>ReleaseRecords { </span><span> </span><span style="color:#bf616a;">releases</span><span>: BTreeSet&lt;Release&gt;, </span><span>} </span><span> </span><span style="color:#65737e;">/// A single release. </span><span style="color:#65737e;">/// </span><span style="color:#65737e;">/// In this example, we define a release as a toolchain of a specific </span><span style="color:#65737e;">/// version (stable, beta) or date (nightly), and its associated </span><span style="color:#65737e;">/// components. </span><span style="color:#b48ead;">struct </span><span>Release { </span><span> </span><span style="color:#bf616a;">toolchain</span><span>: rust_toolchain::Toolchain, </span><span> </span><span style="color:#65737e;">/// Rustup has the concept of components and extensions. </span><span> </span><span style="color:#65737e;">/// </span><span> </span><span style="color:#65737e;">/// When installing a toolchain, components are installed by default, while extensions are optional components. </span><span> </span><span style="color:#65737e;">/// In this implementation, they&#39;re combined. </span><span> </span><span style="color:#bf616a;">components</span><span>: Vec&lt;rust_toolchain::Component&gt;, </span><span>} </span></code></pre> <p>Now, let's consider how we want to use the data captured by these data structures. For example, we may want to find the most recently released Rust release:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">impl </span><span>ReleaseRecords { </span><span> </span><span style="color:#65737e;">/// Find the most recent Rust release. </span><span> </span><span style="color:#65737e;">/// </span><span> </span><span style="color:#65737e;">/// Returns `None` if no release could be found. </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">last_released</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>) -&gt; Option&lt;&amp;Release&gt; { </span><span> todo!() </span><span> } </span><span>} </span></code></pre> <p>See that <code>todo!</code> macro 😃? Instead of providing an actual, or fake, implementation which needs to satisfy the return type of our method, we placed a <code>todo!</code> macro in the body. This allows us to not worry about our implementation just yet, so we can focus on the design of our API instead.</p> <p>It also accepts the same arguments as <a href="https://doc.rust-lang.org/std/macro.panic.html">panic!</a>, so the following will work as well:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">impl </span><span>Release { </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">release_date</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>) -&gt; rust_toolchain::ReleaseDate { </span><span> todo!(&quot;</span><span style="color:#a3be8c;">release date of the toolchain</span><span>&quot;) </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">find_component</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>, </span><span style="color:#bf616a;">name</span><span>: &amp;</span><span style="color:#b48ead;">str</span><span>) -&gt; Option&lt;rust_toolchain::Component&gt; { </span><span> todo!(&quot;</span><span style="color:#a3be8c;">find component with name: &#39;{name}&#39;</span><span>&quot;) </span><span> } </span><span>} </span></code></pre> <p>If we run the <code>find_component</code> method, we'll find that it panics the thread, and shows the panic message we provided, prefixed with "not yet implemented":</p> <pre data-lang="text" style="background-color:#2b303b;color:#c0c5ce;" class="language-text "><code class="language-text" data-lang="text"><span>not yet implemented: find component with name: &#39;hello-world&#39; </span><span>thread &#39;tests::find_component&#39; panicked at &#39;not yet implemented: find component with name: &#39;hello-world&#39;&#39;, crates/rust-releases-core/src/lib.rs:77:9 </span><span>stack backtrace: </span><span> 0: std::panicking::begin_panic_handler </span><span> at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/std/src/panicking.rs:579 </span><span> 1: &lt;snip&gt; </span><span>note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. </span></code></pre> <p>Under the hood, <code>todo!</code> is the same as <code>panic!</code>, to which it defers its implementation, but with a clear change of semantics: this bit is not yet implemented, but we'll do so soon.</p> <h2 id="every-rose-has-its-thorn">Every rose has its thorn</h2> <p>Let's expand our design and add a few more useful methods:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">impl </span><span>Release { </span><span> </span><span style="color:#65737e;">/// Returns an iterator over the components which are installed by default. </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">components</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>) -&gt; impl Iterator&lt;Item = &amp;rust_toolchain::Component&gt; { </span><span> todo!(&quot;</span><span style="color:#a3be8c;">components installed by default</span><span>&quot;) </span><span> } </span><span> </span><span> </span><span style="color:#65737e;">/// Returns an iterator over the components which are optional, </span><span> </span><span style="color:#65737e;">/// and not installed by default. </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">extensions</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>) -&gt; impl Iterator&lt;Item = &amp;rust_toolchain::Component&gt; { </span><span> todo!(&quot;</span><span style="color:#a3be8c;">components not installed by default</span><span>&quot;) </span><span> } </span><span>} </span></code></pre> <p>In the above code we defined two methods on <code>Release</code>, which both return an iterator of <code>&amp;rust_toolchain::Component</code> items. What happens if we try to compile the code above?:</p> <pre data-lang="rust_errors" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust_errors "><code class="language-rust_errors" data-lang="rust_errors"><span>error[E0277]: `()` is not an iterator </span><span> --&gt; crates/rust-releases-core/src/lib.rs:80:33 </span><span> | </span><span>80 | pub fn components(&amp;self) -&gt; impl Iterator&lt;Item = &amp;rust_toolchain::Component&gt; { </span><span> | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `()` is not an iterator </span><span> | </span><span> = help: the trait `Iterator` is not implemented for `()` </span></code></pre> <p>😢</p> <p>It <a href="https://github.com/rust-lang/rust/issues/36375#issuecomment-357216289">turns out</a>, there is an <a href="https://github.com/rust-lang/rust/issues/36375">issue</a> where the compiler is unable to figure out what type to use for types which have the never type as their return type and use <code>impl Trait</code> in return position. The <code>todo!</code> macro falls in this category.</p> <p>There are several ways to work around the problem though.</p> <p>One option is to use dynamic dispatch and box:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">impl </span><span>Release { </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">extensions</span><span>&lt;</span><span style="color:#b48ead;">&#39;this</span><span>&gt;( </span><span> &amp;</span><span style="color:#b48ead;">&#39;this </span><span style="color:#bf616a;">self</span><span>, </span><span> ) -&gt; Box&lt;dyn Iterator&lt;Item = &amp;</span><span style="color:#b48ead;">&#39;this </span><span>rust_toolchain::Component&gt; + </span><span style="color:#b48ead;">&#39;this</span><span>&gt; { </span><span> todo!(&quot;</span><span style="color:#a3be8c;">components not installed by default</span><span>&quot;) </span><span> } </span><span>} </span></code></pre> <p>Another is to return a simple concrete type:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">impl </span><span>Release { </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">extensions</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>) -&gt; Vec&lt;&amp;rust_toolchain::Component&gt; { </span><span> todo!(&quot;</span><span style="color:#a3be8c;">components not installed by default</span><span>&quot;) </span><span> } </span><span>} </span></code></pre> <p>A third is to use the explicit type in the return type, but this requires you to think ahead on which iterator type you will be using, which you probably don't want to worry about (especially if you will be using <code>impl Iterator</code> on implementation). A fourth would be to implement <code>Iterator</code> for a concrete type, but this also adds a lot of boilerplate.</p> <p>A <a href="https://reddit.com/r/rust/comments/12x1lfd/blog_post_using_the_todo_macro_to_prototype_your/jhjecab/">fifth</a> was suggested by <em>natalialt</em>. I like this one very much, because it is short and to the point (altough also not universably applicable to any <code>impl Trait</code> return type). This solution returns an iterator via the <code>std::iter::once()</code> function:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">use </span><span>std::iter; </span><span> </span><span style="color:#b48ead;">impl </span><span>Release { </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">extensions</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>) -&gt; impl Iterator&lt;Item = &amp;rust_toolchain::Component&gt; { </span><span> iter::once(todo!(&quot;</span><span style="color:#a3be8c;">components not installed by default</span><span>&quot;)) </span><span> } </span><span>} </span></code></pre> <p>The reason that this works, is because <code>std::iter::once</code> returns a valid iterator, namely <a href="https://doc.rust-lang.org/std/iter/struct.Once.html">Once</a>, which makes the return type satisfiable. Where the never type <code>!</code> didn't implement <code>Iterator</code>, <code>Once</code> does.</p> <h2 id="taking-it-to-the-next-level">Taking it to the next level</h2> <p>So far, we saw that <code>todo!</code> can be a powerful prototyping tool. If we want to take it to the next level, we should start making use of Rust's type checking capabilities for the composition and usage of our API. This helps us be much more confident throughout designing of the library.</p> <p>As shown before, the compiler knows that we do not need to satisfy our return types (within the method body). However, the return type of these methods will still be checked wherever we use these methods. This allows us to not only define the new API, while leaving the implementation for later but also write some code on how to use it. This can be particularly useful to explore whether an API is easy to use as a caller.</p> <p>I like to do this by writing unit tests for my API. If your API is painful to use, you're much more likely to find this out if you had to write a usage example yourself. Plus, this way you also already have a first test in place. Example:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">test</span><span>] </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">find_component_returns_none_if_release_has_no_components</span><span>() { </span><span> </span><span style="color:#b48ead;">let</span><span> channel = rust_toolchain::Channel::Nightly; </span><span> </span><span style="color:#b48ead;">let</span><span> release_date = rust_toolchain::ReleaseDate::new(</span><span style="color:#d08770;">2023</span><span>, </span><span style="color:#d08770;">1</span><span>, </span><span style="color:#d08770;">1</span><span>); </span><span> </span><span style="color:#b48ead;">let</span><span> platform = rust_toolchain::Platform::host(); </span><span> </span><span style="color:#b48ead;">let</span><span> version = None; </span><span> </span><span> </span><span style="color:#b48ead;">let</span><span> toolchain = rust_toolchain::Toolchain::new(channel, release_date, platform, version); </span><span> </span><span> </span><span style="color:#b48ead;">let</span><span> release = Release::new(toolchain, vec![]); </span><span> </span><span style="color:#b48ead;">let</span><span> component = release.</span><span style="color:#96b5b4;">find_component</span><span>(&quot;</span><span style="color:#a3be8c;">hello</span><span>&quot;); </span><span> </span><span> assert!(component.</span><span style="color:#96b5b4;">is_none</span><span>()); </span><span>} </span></code></pre> <p>And if we were not yet sure how to construct an input for our function under test, we can also use the <code>todo!</code> macro here:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">test</span><span>] </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">find_component_returns_none_if_release_has_no_components</span><span>() { </span><span> </span><span style="color:#65737e;">// We can use todo!() here too! </span><span> </span><span style="color:#65737e;">// </span><span> </span><span style="color:#65737e;">// We may not be sure yet how to construct our input. </span><span> </span><span style="color:#65737e;">// Let&#39;s take the design of the API, one step at a time. </span><span> </span><span style="color:#b48ead;">let</span><span> toolchain = todo!(); </span><span> </span><span> </span><span style="color:#65737e;">// The code below will be unreachable though! </span><span> </span><span style="color:#b48ead;">let</span><span> release = Release::new(toolchain, vec![]); </span><span> </span><span style="color:#b48ead;">let</span><span> component = release.</span><span style="color:#96b5b4;">find_component</span><span>(&quot;</span><span style="color:#a3be8c;">hello</span><span>&quot;); </span><span> </span><span> assert!(component.</span><span style="color:#96b5b4;">is_none</span><span>()); </span><span>} </span></code></pre> <p>Alternatively, instead of writing inline unit tests, you could also use <a href="https://doc.rust-lang.org/rustdoc/write-documentation/documentation-tests.html">doctests</a> for this purpose:</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span style="color:#b48ead;">impl </span><span>Release { </span><span> </span><span style="color:#65737e;">/// Find a component by its name. </span><span> </span><span style="color:#65737e;">/// </span><span> </span><span style="color:#65737e;">/// If the component does not exist for this `Release`, </span><span> </span><span style="color:#65737e;">/// returns `Option::None`. </span><span> </span><span style="color:#65737e;">/// </span><span> </span><span style="color:#65737e;">/// # Example </span><span> </span><span style="color:#65737e;">/// </span><span> </span><span style="color:#65737e;">/// ```rust </span><span> </span><span style="color:#65737e;">/// use rust_releases_core::Release; </span><span> </span><span style="color:#65737e;">/// </span><span> </span><span style="color:#65737e;">/// let channel = rust_toolchain::Channel::Nightly; </span><span> </span><span style="color:#65737e;">/// let release_date = rust_toolchain::ReleaseDate::new(2023, 1, 1); </span><span> </span><span style="color:#65737e;">/// let platform = rust_toolchain::Platform::host(); </span><span> </span><span style="color:#65737e;">/// let version = None; </span><span> </span><span style="color:#65737e;">/// </span><span> </span><span style="color:#65737e;">/// let toolchain = rust_toolchain::Toolchain::new(channel, release_date, platform, version); </span><span> </span><span style="color:#65737e;">/// </span><span> </span><span style="color:#65737e;">/// let release = Release::new(toolchain, vec![]); </span><span> </span><span style="color:#65737e;">/// let component = release.find_component(&quot;hello&quot;); </span><span> </span><span style="color:#65737e;">/// </span><span> </span><span style="color:#65737e;">/// assert!(component.is_none()); </span><span> </span><span style="color:#65737e;">/// ``` </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">find_component</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>, </span><span style="color:#bf616a;">name</span><span>: &amp;</span><span style="color:#b48ead;">str</span><span>) -&gt; Option&lt;&amp;rust_toolchain::Component&gt; { </span><span> todo!(&quot;</span><span style="color:#a3be8c;">find component with name: &#39;{name}&#39;</span><span>&quot;) </span><span> } </span><span>} </span></code></pre> <h2 id="implementation-time">Implementation time!</h2> <p>Once we are satisfied with the basic structure of our API, we can gradually replace each <code>todo!</code> macro with an actual implementation. We do not have to replace all the macros simultaneously, so we can focus on one implementation step at a time. Developing a well-designed API requires careful planning and attention to detail. Taking the time to establish a solid foundation will pay off in the long run, as it will result in more user-friendly and reliably designed API.</p> <h1 id="footnotes">Footnotes</h1> <p><sup><span id="footnote1">1</span></sup> I'm currently working on the next version of <a href="https://github.com/foresterre/rust-releases">rust-releases</a>.</p> <h1 id="thanks">Thanks!</h1> <p>Special thanks to Chris Langhout, Jean de Leeuw and Martijn Steenbergen for proofreading my blog post; any mistakes are solely mine.</p> <p>Also many thanks to <em>proudHaskeller</em> on <a href="https://reddit.com/r/rust/comments/12x1lfd/blog_post_using_the_todo_macro_to_prototype_your/jhhn8na/">Reddit</a> for reporting an issue I missed: the type signature I initially used to deal with the <code>todo!</code> and <code>impl Trait</code> would never type check with a concrete implementation (this has been addressed), and to <em>natalialt</em> on <a href="https://reddit.com/r/rust/comments/12x1lfd/blog_post_using_the_todo_macro_to_prototype_your/jhjecab/">Reddit</a> for suggesting a useful workaround to the same issue when the return type is <code>impl Iterator</code>, by using <code>iter::once(todo!())</code>.</p> <h1 id="discuss">Discuss</h1> <p>Discussions and feedback are most welcome! Discuss on <a href="https://reddit.com/r/rust/comments/12x1lfd/blog_post_using_the_todo_macro_to_prototype_your/">Reddit</a>, <a href="https://news.ycombinator.com/item?id=35683396">HackerNews</a> or create an <a href="https://github.com/foresterre/foresterre.github.io/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc">issue</a>.</p>