Dr Tom Palmer https://remlapmot.github.io/ Dr Tom Palmer Source Themes Academic (https://sourcethemes.com/academic/)en-usThu, 12 Feb 2026 00:00:00 +0000 https://remlapmot.github.io/images/icon_hu_4c69fe6e68a3b4.png Dr Tom Palmer https://remlapmot.github.io/ How Posit's Public Package Manager manylinux_2_28 repository can help you if your R project is stuck on Ubuntu Focal Fossa https://remlapmot.github.io/post/2026/manylinux-packages/ Thu, 12 Feb 2026 00:00:00 +0000 https://remlapmot.github.io/post/2026/manylinux-packages/ <h2 id="introduction">Introduction</h2> <p>I am a massive fan of repositories making binary R packages available. This includes the canonical CRAN repositories, <a href="https://r-universe.dev/" target="_blank" rel="noopener">r-universe</a>, <a href="https://eddelbuettel.github.io/r2u/" target="_blank" rel="noopener">r2u</a>, and the <a href="https://packagemanager.posit.co/" target="_blank" rel="noopener">Posit (Public) Package Manager</a>, and there are others. R-universe is outstanding because it builds binaries of GitHub only packages. The Posit Public Package Manager is outstanding due to its incredible breadth (it makes binaries for 14 Linux distros) and also its depth (its almost daily snapshotting service is remarkably useful for quickly making reproducible R environments).</p> <p>Today I wanted to highlight how the manylinux_2_28 packages in the Posit Public Package Manager helped me out. Posit released this in <a href="https://packagemanager.posit.co/cran/__linux__/manylinux_2_28/latest" target="_blank" rel="noopener">June 2025</a>. Ironically, I was using another Posit service, Posit Cloud (formerly RStudio Cloud). Within this I have quite a few projects in workspaces which are over 3 years old. Behind the scenes these are running on Ubuntu Focal Fossa Linux. I believe that if I create a new Posit Cloud RStudio project that will be running on Ubuntu Noble Numbat and so users of new projects won&rsquo;t need this tip.</p> <p>Unfortunately, Focal Fossa is out of support (except if you have Ubuntu Pro) and hence Posit have removed their Focal repository packages from the Posit Public Package Manager, which is fair enough. Within the Posit Cloud project workspace Posit kindly make available a private version of what I think was that Focal repository. However, for reasons I don&rsquo;t fully understand a fair number of the packages I required weren&rsquo;t built as binaries.</p> <p>That got me thinking, could the manylinux_2_28 packages help here? In the name the 2.28 refers to the minimum version of the glibc library that the Linux distro needs to come with. I realised that I didn&rsquo;t know what version of glibc Ubuntu Focal Fossa came with. A quick</p> <pre><code class="language-sh">ldd --version </code></pre> <p>revealed</p> <pre><code class="language-plaintext">ldd (Ubuntu GLIBC 2.31-0ubuntu9.17) 2.31 Copyright (C) 2020 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper. </code></pre> <p>And hence I was in luck as version 2.31 is after 2.28. Therefore, in my Posit Cloud workspace I simply switched my syntax to installing packages to</p> <pre><code class="language-r">install.packages( 'tidyverse', repos = 'https://packagemanager.posit.co/cran/__linux__/manylinux_2_28/latest' ) </code></pre> <p>and all the packages came in as binaries. If you are not running this from within RStudio your syntax would need to be the following as per <a href="https://packagemanager.posit.co/client/#/repos/cran/setup" target="_blank" rel="noopener">the setup page</a></p> <pre><code class="language-r">options(repos = c( CRAN = sprintf(&quot;https://packagemanager.posit.co/cran/latest/bin/linux/manylinux_2_28-%s/%s&quot;, R.version[&quot;arch&quot;], substr(getRversion(), 1, 3)) ) ) install.packages('tidyverse') </code></pre> <img src="https://remlapmot.github.io/post/2026/manylinux-packages/img/installing-manylinux-packages.png" alt="Screenshot of installing packages in a Posit Cloud RStudio project running on Ubuntu Focal Fossa." width="630" style="display: block; margin: auto;"> <p>My other solution could have been to create a new RStudio project, which as I said would be running on Noble Numbat.</p> <h2 id="summary">Summary</h2> <p>In summary, manylinux_2_28 binary packages in the Posit Public Package Manager can be used in Ubuntu Focal Fossa. Thanks again to Posit for this great resource.</p> My #GitHubUnwrapped 2025! https://remlapmot.github.io/post/2025/github-unwrapped-2025/ Fri, 05 Dec 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/github-unwrapped-2025/ <p>My #GitHubUnwrapped 2025!</p> <p align="center"><iframe src="https://drive.google.com/file/d/1SE5-CKtZDJEEghE8zJz-r3UlW9J8dhDC/preview" width="640" height="480" allow="autoplay"></iframe></p> <p>Made with <a href="https://www.githubunwrapped.com/" target="_blank" rel="noopener">https://www.githubunwrapped.com/</a>.</p> Executable Python scripts with inline metadata and why friends don't let friends install R packages with remotes::install_github() https://remlapmot.github.io/talk/2025_prog1/ Wed, 10 Sep 2025 14:00:00 +0000 https://remlapmot.github.io/talk/2025_prog1/ Creating self-contained executable Python scripts for rendering Quarto documents using the Jupyter engine https://remlapmot.github.io/post/2025/self-contained-python-script-for-quarto/ Tue, 02 Sep 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/self-contained-python-script-for-quarto/ <h2 id="introduction">Introduction</h2> <p>In previous posts I have covered creating <a href="https://remlapmot.github.io/post/2025/multi-engine-quarto/" target="_blank" rel="noopener">effectively multi-engine Quarto documents</a> and also how to use <a href="https://remlapmot.github.io/post/2025/nbstata-uv-venv/" target="_blank" rel="noopener">uv virtual environments for the nbstata Jupyter kernel</a> to run Quarto documents with Stata code. Hence by trivial extension we can use <code>uv</code> to manage the virutal environments for running Quarto documents using the <code>jupyter: python3</code> engine.</p> <p>The slight inconvenience about this approach when you are working on a collaborative project with new Python users is that you end up leaving a README or shell script with the required commands to activate the environment, install the nbstata kernel, and run Quarto. I sense this management of the virutal environment is a pain point for new Python users - who are likely wondering what on earth a virtual environment is. So I have been looking for a way to simplify this process for them.</p> <p>A recent <a href="https://www.rostrum.blog/posts/2025-08-11-uv-standalone/" target="_blank" rel="noopener">post by Matt Dray</a> about using uv to run self-contained executable Python scripts got me thinking. Could I produce a similar self-contained executable Python script to perform the rendering for Quarto documents using the Jupyter engine. Then my colleagues would only need to call the script as an executable at the command line. This would avoid them the trouble of managing the virtual environment.</p> <h2 id="the-self-contained-executable-python-script">The self-contained executable Python script</h2> <p>I came up with the following Python script.</p> <pre><code class="language-python">#!/usr/bin/env -S uv run --script # /// script # requires-python = &quot;&gt;=3.9&quot; # dependencies = [ # &quot;jupyterlab&gt;=4.4.3&quot;, # &quot;jupyterlab-stata-highlight2&gt;=0.1.2&quot;, # &quot;nbstata&gt;=0.8.3&quot;, # ] # /// import subprocess cmd0 = &quot;python -m nbstata.install --sys-prefix&quot; retval0 = subprocess.call(cmd0, shell=True) print('returned value:', retval0) cmd1 = &quot;quarto render --profile stata-questions&quot; retval1 = subprocess.call(cmd1, shell=True) print('returned value:', retval1) cmd2 = &quot;quarto render --profile stata-solutions&quot; retval2 = subprocess.call(cmd2, shell=True) print('returned value:', retval2) </code></pre> <ul> <li> <p>The first line, the shebang ensures it is run by <code>uv run</code></p> </li> <li> <p>The metadata defining the Python environment is declared between the</p> <pre><code class="language-python"># /// script # ... # /// </code></pre> <ul> <li>If you don&rsquo;t use Stata, say you are using the <code>jupyter: python3</code> engine then you can delete the <code>jupyterlab-stata-highlight2</code> and <code>nbstata</code> entries and the first group of 3 lines for <code>cmd0</code>.</li> <li>If you use additional Python packages in your code then you need to add them to the list.</li> </ul> </li> <li> <p>Then comes the actual code. These are simply system calls using the <em>subprocess</em> module. You can amend the number of calls and the calls themselves inside the string quotes as required. I am recreating some of the rendering commands in my recent <a href="https://remlapmot.github.io/post/2025/quarto-profiles-for-tutorials/" target="_blank" rel="noopener">post about using Quarto profiles for tutorial documents</a>. It&rsquo;s worth pointing out that I haven&rsquo;t used the Python quarto package here as it&rsquo;s currently slightly too limited for my use (I&rsquo;m not sure it can render profiles).</p> </li> </ul> <p>Save the script in a file, say <em>render</em>, then make it executable with</p> <pre><code class="language-sh">chmod +x render </code></pre> <p>Then all my colleagues need to do is run it with</p> <pre><code class="language-sh">./render </code></pre> <p>Of course uv and Quarto need to be installed and be on their <code>PATH</code>, and Stata needs to be installed locally when using that. For my colleagues using Windows, they need to run this from a Git Bash shell rather than from Powershell or CMD shell (for the shebang line to work).</p> <p>If you are only using the Quarto knitr engine then you don&rsquo;t need this script because you don&rsquo;t need Jupyter.</p> <p>And for more information about uv Python scripts, the full documentation is <a href="https://docs.astral.sh/uv/guides/scripts/#creating-a-python-script" target="_blank" rel="noopener">here</a>.</p> <h2 id="summary">Summary</h2> <p>I have shown how to make a self-contained executable Python script to render Quarto documents using the Jupyter engine which automatically manage their own virtual environment. This means users don&rsquo;t have to manage the virtual environment themselves, which can be a pain point for new Python users.</p> Seven accessibility tips for Quarto and R Markdown users https://remlapmot.github.io/post/2025/quarto-accessibility/ Fri, 15 Aug 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/quarto-accessibility/ <h2 id="introduction">Introduction</h2> <p>When teaching, for my practicals/tutorials and for about half of my lectures I find myself preparing them using R Markdown and laterly Quarto. I enjoy preparing the material in R Markdown and Quarto because it gives me a reproducible way of regenerating my material every year and I can track changes in the Rmd/qmd files very precisely with Git.</p> <p>Recently, the subject of accessibility has become more prominent within Universities. In the UK, and in many other countries, we are legally obliged to produce accessible learning materials that do not disadvantage disabled students.</p> <p>My University uses Blackboard for its online learning environment (OLE)/learning management system (LMS). There are other OLEs, e.g., Moodle, Canvas, etc. They all work in essentially the same way in that they provide a website per Unit/Module/Course within a secure online system.</p> <p>Every document I upload into Blackboard receives an accessibility score (out of 100%) and each module I teach receives an overall accessibility score. My University uses <a href="https://ally.ac/" target="_blank" rel="noopener">Anthology Ally</a> Accessibility Report to generate these scores. My university doesn&rsquo;t have a rule about what&rsquo;s an acceptable score for either a document or a course.</p> <p>It turns out accessibility is sometimes abbreviated to a11y, which like k8s (for Kubernetes), is a numeronym, where the 11 stands for the 11 letters in between the starting <em>a</em> and the ending <em>y</em> of <em>accessibility</em>.</p> <p>I should say there are many guides to accessibility for HTML documents online, and indeed for R and R Markdown there are at least two packages on CRAN, <a href="https://cran.r-project.org/package=accessrmd" target="_blank" rel="noopener"><strong>accessrmd</strong></a> and <a href="https://paulnorthrop.github.io/accessr/" target="_blank" rel="noopener"><strong>accessr</strong></a>, addressing accessibility issues. Also I have not covered topics such as choosing an accessible color palette in say <strong>ggplot2</strong>.</p> <p>What follows is a set of tips which help improve the accessibility score for individual documents and hence your overall module accessibility score.</p> <h2 id="tip-1-replace-all-pdfs-with-docx-pptx-or-html-documents">Tip 1: Replace all pdfs with docx, pptx, or html documents</h2> <p>For a pdf to get a high accessibility score it needs to be a special type of pdf called a &rsquo;tagged pdf&rsquo;, otherwise it will get a very low score (approx. 6%).</p> <p>As I will show in the following tips, it is much easier to make Word, Powerpoint, and html documents accessible. If you are looking for the most effective boost to your accessibility scores simply remove all pdf documents from your site and replace them with Word, Powerpoint, or html documents.</p> <h2 id="tip-2-add-alt-text-to-all-figures">Tip 2: Add alt text to all figures</h2> <p>I guess like many R users the first I remember hearing about accessibility was that there is this thing called alt text and it&rsquo;s best practice for html documents to provide alt text summaries of every image they contain. Then screen readers have a description of the image for visually impaired readers.</p> <p>It turns out that other types of document can also hold alt text for images; including Word, Powerpoint, and pdf documents.</p> <p>Let&rsquo;s say we have a Quarto (or R Markdown) document which we render to docx and html output formats. To add alt text to figures in the html document we can use the <code>fig-alt</code> chunk option (<code>fig.alt</code> in R Markdown). However, it turns out that alt text for a Word document is taken from the <code>fig-cap</code> chunk option (<code>fig.cap</code> in R Markdown). Therefore, I specify both <code>fig-alt</code> and <code>fig-cap</code> chunk options in all code chunks generating figures.</p> <pre><code class="language-plaintext">```{r} #| fig-cap: Kaplan-Meier survival curve. #| fig-alt: Plot of a Kaplan-Meier survival curve. # code to generate plot ``` </code></pre> <h2 id="tip-3-the-surprising-headache-that-is-creating-tagged-pdfs-">Tip 3: (The surprising headache that is) Creating tagged pdfs ๐Ÿ˜ฌ๐Ÿคฏ</h2> <p>It is possible for a pdf to obtain a perfect accessibility score; but as I said above it must be a tagged pdf.</p> <p>You can check if a pdf has tags by opening it in Adobe Acrobat Reader then bringing up the document properties window. The information is in the bottom left of the <em>Description</em> tab (on Windows I found that SumatraPDF also reported that information but on macOS I found I couldn&rsquo;t see this reported in the Preview app).</p> <img src="https://remlapmot.github.io/post/2025/quarto-accessibility/img/adobe-reader-pdf-info.png" alt="Screenshot of document properties tab in Adobe Acrobat Reader." width="550" style="display: block; margin: auto;"> <p>It turns out that we can create tagged pdfs in a few ways;</p> <ul> <li> <p>by exporting a Word or Powerpoint document to pdf within those programs</p> <ul> <li>note when exporting to pdf on macOS, users must select the <em>Best for electronic distribution and accessibility&hellip;</em> option (the <em>Best for printing</em> option does not generate tagged pdfs) <img src="https://remlapmot.github.io/post/2025/quarto-accessibility/img/powerpoint-macos-pdf-export.png" alt="Screenshot of Powerpoint export to pdf window on macOS." width="630" style="display: block; margin: auto;"></li> <li>Word and Powerpoint for Windows and Word and Powerpoint online export tagged pdfs by default</li> </ul> </li> <li> <p>by printing html documents to pdf (Print | Save as pdf) in Chrome (and the other Chromium based browsers such as Edge)</p> <ul> <li>In my testing I find that by default Safari and Firefox do not generate tagged pdfs</li> </ul> </li> <li> <p>The paid for professional version of Adobe Acrobat allows users to <a href="https://helpx.adobe.com/uk/acrobat/using/editing-document-structure-content-tags.html" target="_blank" rel="noopener">add tags to pdfs</a>, but I don&rsquo;t have that, and I assuming you don&rsquo;t either.</p> </li> <li> <p>And I should note that within Blackboard itself Ally can generate a tagged pdf from Word and Powerpoint documents, under the <em>Download Alternative Formats</em> option for a file. <img src="https://remlapmot.github.io/post/2025/quarto-accessibility/img/download-alternative-formats-1.png" alt="Screenshot of selecting to download alternative document formats in Blackboard." width="630" style="display: block; margin: auto;"></p> <img src="https://remlapmot.github.io/post/2025/quarto-accessibility/img/download-alternative-formats-2.png" alt="Screenshot of the alternative document formats in Blackboard." width="450" style="display: block; margin: auto;"> </li> </ul> <p>The surprise here is that pdf documents produced by LaTeX and typst by default do not (yet) generate tagged pdfs. Therefore, do not render to <code>pdf</code>/<code>typst</code> output formats in Quarto nor <code>pdf_document</code> in R Markdown (nor <code>pdf_document2</code> in bookdown).</p> <h3 id="generating-tagged-pdfs-in-latex">Generating tagged pdfs in LaTeX</h3> <p>It turns out it <a href="https://latex3.github.io/tagging-project/documentation/prototype-usage-instructions" target="_blank" rel="noopener">is possible to generate a tagged pdf from LaTeX</a>, but this is extrememly inconvenient from R Markdown and Quarto. First use Quarto/R Markdown to generate the TeX file of your document, e.g. in <a href="https://quarto.org/docs/output-formats/pdf-basics.html#latex-output" target="_blank" rel="noopener">Quarto specify either</a></p> <pre><code class="language-yaml">format: pdf: keep-tex: true </code></pre> <p>or the latex output format.</p> <pre><code class="language-yaml">format: latex </code></pre> <p>You then require <a href="https://tex.stackexchange.com/a/605142" target="_blank" rel="noopener">the following LaTeX packages</a>.</p> <pre><code class="language-r">tinytex::tlmgr_install(c('latex-lab', 'pdfmanagement-testphase', 'tagpdf', 'luamml')) </code></pre> <p>You then need to amend the very top of your TeX file to have a <code>\DocumentMetadata{tagging=on}</code> entry. And possibly using the <em>unicode-math</em> package is helpful. The beginning of your TeX file will look like the following.</p> <pre><code class="language-latex">\DocumentMetadata{tagging=on} \documentclass{article} \usepackage{unicode-math} \begin{document} % the rest of your document ... \end{document} </code></pre> <p>When I tried this I found that compiling with LuaLaTeX did generate a tagged pdf.</p> <pre><code class="language-sh">lualatex mydocument.tex </code></pre> <p>But I cannot face going through this hassle for every document I produce. So the slightly unexpected take home message here is that if you want to give your students a pdf document, the most convenient way to produce a tagged pdf is to render your Quarto document to docx or html and then export the pdf from within either Word or Chrome or let Ally do the conversion for you.</p> <h2 id="tip-4-improving-table-accessibility">Tip 4: Improving table accessibility</h2> <p>I find that tables generated using the Markdown syntax generate warnings in Ally Accessibility Checker. It reports it can&rsquo;t find the header row. However, I find that tables generated using the <strong>gt</strong> package, and other table packages usually pass. So I avoid creating tables using Markdown syntax.</p> <h2 id="tip-5-different-accessibility-checkers-can-report-different-results">Tip 5: Different accessibility checkers can report different results</h2> <p>Microsoft has built an accessibility checker into alot of its Office suite. You can access this in Word and Powerpoint from the <em>Review</em> tab then <em>Check Accessibility</em>.</p> <img src="https://remlapmot.github.io/post/2025/quarto-accessibility/img/microsoft-ribbon-accessibility.png" alt="Screenshot of accessing the accessibility checker within Microsoft Word." width="630" style="display: block; margin: auto;"> <p>Compared to Ally Accessibility Checker I find this checks for more conditions. For the items they both check I find that most of the time they agree. But sometimes they don&rsquo;t agree, e.g., the Microsoft checker reported that the slide numbers in a Powerpoint deck were ok, whereas Ally Accessibility Checker reported that they failed its colour contrast check (even though they were black text on a white slide).</p> <h2 id="tip-6-improving-the-accessibility-of-custom-word-document-templates">Tip 6: Improving the accessibility of custom Word document templates</h2> <p>If you want to use a <a href="https://bookdown.org/yihui/rmarkdown-cookbook/word-template.html" target="_blank" rel="noopener">custom template Word document</a> for your docx output it needs to be based on the underlying Pandoc docx template. You can obtain that by running</p> <pre><code class="language-sh">pandoc --print-default-data-file reference.docx &gt; reference.docx </code></pre> <p>You can then adjust the formatting of the various styles in Word in the <em>Styles Pane</em>, and also if you are in the UK/EU you can amend the layout size from US Letter paper to A4.</p> <p>Then run the Microsoft Accessibilty checker on this document (<em>Review</em> tab | <em>Check Accessibility</em>) and Word will give you the option to upgrade the format of the document so that it can run, so click the <em>Convert</em> button when prompted.</p> <img src="https://remlapmot.github.io/post/2025/quarto-accessibility/img/word-accessibility-checker-convert-format.png" alt="Screenshot of Microsoft Word acccessibility checker convert document." width="350" style="display: block; margin: auto;"> <p>Then if you&rsquo;ve added an image to the document header you can add alt text to it. Then resave the document. Then specify the use of the reference document in your Quarto YAML header.</p> <pre><code class="language-yaml">format: docx: reference-doc: reference.docx </code></pre> <h2 id="tip-7-uploading-quartocomplex-html-documents-into-blackboard-ultra">Tip 7: Uploading Quarto/complex html documents into Blackboard Ultra</h2> <p>My university has upgraded to the latest version of Blackboard - Ultra. Prior to this upgrade I had no trouble uploading any type of document. But now if I try to upload html documents with embedded resources generated from Quarto the upload animation hangs (the 3 squares on the right of the screenshot below simply keep animating) and the file is not uploaded.</p> <img src="https://remlapmot.github.io/post/2025/quarto-accessibility/img/upload-hang.png" alt="Screenshot of Blackboard file upload hanging." width="750" style="display: block; margin: auto;"> <p>It turns out this is because Blackboard scans every document we upload, which it calls its content sanitization check. For some reason Quarto html documents, including revealjs html slide decks, are now too complex and fail this check. Annoyingly, Blackboard does not emit any error message to tell us this. However, there is a workaround.</p> <ul> <li>First, put your html file into a zip archive</li> <li>Then from within your module site, go to <em>Content Collection</em> when adding an item <img src="https://remlapmot.github.io/post/2025/quarto-accessibility/img/add-course-content-from-within-site.png" alt="Screenshot of selecting Course Content in Blackboard whn uploading a file." width="630" style="display: block; margin: auto;"></li> <li>Then select <em>Upload</em> | <em>Upload Zip Package</em> (choosing any other option will fail) <img src="https://remlapmot.github.io/post/2025/quarto-accessibility/img/blackboard-upload-zip-package.png" alt="Screenshot of uploading a zip package of a Quarto html file." width="630" style="display: block; margin: auto;"></li> <li>Make sure to select the button to overwrite an existing file with the same name</li> <li>And click <em>Submit</em> and that&rsquo;s it.</li> </ul> <p>This is the only way I can find to bypass the content sanitization check. Note if you attempt to upload the zip archive from your <em>Content Collection</em> accessed not from within the course site - as per screenshot below (don&rsquo;t go here!) - the upload will also fail (confusing I know!).</p> <img src="https://remlapmot.github.io/post/2025/quarto-accessibility/img/blackboard-content-collection.png" alt="Screenshot of Blackboard Content Collection selection not from within a course." width="750" style="display: block; margin: auto;"> <h2 id="summary">Summary</h2> <p>I have described 7 hopefully helpful tips about how to improve the accessibility scores for documents produced from Quarto and R Markdown. Using these tips I have increased the accessibility score for my module from 67% for last year&rsquo;s site to 98% for this year&rsquo;s site.</p> <img src="https://remlapmot.github.io/post/2025/quarto-accessibility/img/summary-score.png" alt="Screenshot of Ally Accessibility Checker summary score for my module within Blackboard online learning environment." width="630" style="display: block; margin: auto;"> <p>I hope that you have similar success.</p> Creating tutorial worksheets: Quarto profiles for the win! https://remlapmot.github.io/post/2025/quarto-profiles-for-tutorials/ Sun, 06 Jul 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/quarto-profiles-for-tutorials/ <h2 id="introduction">Introduction</h2> <p>I <a href="https://remlapmot.github.io/post/2025/quarto-conditional-content/" target="_blank" rel="noopener">previously posted</a> about creating tutorial worksheets for 4 different Quarto engines (for R, Python, Stata, and Julia) using <a href="https://quarto.org/docs/authoring/conditional.html" target="_blank" rel="noopener">conditional content</a>. However, that approach is a bit hacky and I wasn&rsquo;t very happy with it.</p> <p>Thanks to a <a href="https://quarto-dev.github.io/quarto-r/articles/dynamic-metadata.html" target="_blank" rel="noopener">vignette by Christophe Dervieux</a> in the quarto R package I realised there is a more convenient, and less hacky, way to create tutorial documents using Quarto profiles. I don&rsquo;t use the dynamic metadata approach in the vignette to achieve my solution but it led me to the relevant Quarto documentation page and I discuss it at the end of this post.</p> <p>My aim is the following</p> <blockquote> <p>To have a single Quarto document from which both the question and solution documents can be rendered for a tutorial.</p> </blockquote> <p>As a reminder, as <a href="https://remlapmot.github.io/post/2025/quarto-conditional-content/#introduction" target="_blank" rel="noopener">I showed in my previous post</a>, this has been possible to achieve for a long time with R Markdown and knitr thanks to the brilliant work of Yihui Xie, because they allow programmatic chunk options. Therefore, implementing this in a Quarto document using the knitr engine is also straightforward and I won&rsquo;t repeat it in this post. However, it is the three other Quarto engines (for Python, Stata, and Julia) that I am interested in which are problematic because as far as I know they don&rsquo;t allow programmatic chunk options and they don&rsquo;t have the equivalent of the <code>! expr ...</code> YAML tag literal.</p> <h2 id="quarto-profiles">Quarto profiles</h2> <p>Quarto profiles are introduced <a href="https://quarto.org/docs/projects/profiles.html" target="_blank" rel="noopener">on this page of the Quarto documentation</a>. Specifically, we shall make use of <a href="https://quarto.org/docs/projects/profiles.html#profile-content" target="_blank" rel="noopener">conditional content dependent upon profile</a>.</p> <p>First we will create a simple default Quarto profile file <em>_quarto.yml</em> which will simply contain the following.</p> <pre><code class="language-yml">execute: eval: true </code></pre> <p>(This is because I usually have an R version of the tutorial in the same directory using programmatic chunk options, so I don&rsquo;t want to set a default that is language specific nor will affect anything in the R Quarto documents.)</p> <p>Next for each language I will make profiles for the questions output and the solutions output in appropriately named YAML files (I just show the Python files as the other two just have Stata/Julia substituted in the appropraite places).</p> <h3 id="questions-and-solutions-profiles-for-python">Questions and solutions profiles for Python</h3> <p><em>_quarto-python-questions.yml</em></p> <pre><code class="language-yml">project: render: - tutorial-python.qmd title: 'Questions document: Python version' execute: eval: false format: html: output-file: &quot;tutorial-python-questions&quot; output-ext: &quot;html&quot; </code></pre> <p><em>_quarto-python-solutions.yml</em></p> <pre><code class="language-yml">project: render: - tutorial-python.qmd title: 'Solutions document: Python version' format: html: output-file: &quot;tutorial-python-solutions&quot; output-ext: &quot;html&quot; </code></pre> <p>Then our tutorial Quarto Python document, <em>tutorial-python.qmd</em>, will look like the following.</p> <pre><code class="language-plaintext">--- format: html: embed-resources: true jupyter: python3 --- ## Question 1 Question text. ```{python} print(&quot;The code which is echoed in questions and evaluated in solutions.&quot;) ``` ::: {.content-visible when-profile=&quot;python-solutions&quot;} The text for the solutions. ::: </code></pre> <p>We see the use of the conditional content based upon profile for the text of the solutions, and we could included additional code chunks in these conditional content divs.</p> <p>We repeat this for the other 2 tricky Quarto engines, <code>engine: julia</code> and <code>jupyter: nbstata</code>, including making profile yaml files for each engine and the respective <em>tutorial-{stata/julia}.qmd</em> Quarto documents.</p> <p>Then we make a shell script with our render commands.</p> <pre><code class="language-sh">quarto render --profile python-questions quarto render --profile python-solutions quarto render --profile stata-questions quarto render --profile stata-solutions quarto render --profile julia-questions quarto render --profile julia-solutions </code></pre> <p>And because I have an R version using parameters my shell script usually begins.</p> <pre><code class="language-sh">quarto render tutorial-r.qmd -P solutions:false -o tutorial-r-questions.html quarto render tutorial-r.qmd -o tutorial-r-solutions.html </code></pre> <p>And that&rsquo;s it.</p> <p>You can find the full source code in my example repo <a href="https://github.com/remlapmot/tutorial-quarto-profiles" target="_blank" rel="noopener">here</a> and their rendered output can be viewed from <a href="https://remlapmot.github.io/tutorial-quarto-profiles/" target="_blank" rel="noopener">here</a>. This repo also contains a tutorial document including the 4 languages in the same document using the embed shortcode as I described in <a href="https://remlapmot.github.io/post/2025/multi-engine-quarto/" target="_blank" rel="noopener">another previous post</a>. A screenshot of the questions and solutions documents from this approach <a href="#top">is shown above</a>.</p> <h2 id="an-honorable-mention-about-dynamic-metadata">An honorable mention about dynamic metadata</h2> <p>When I started reading the <a href="https://quarto-dev.github.io/quarto-r/articles/dynamic-metadata.html" target="_blank" rel="noopener">quarto R package vignette</a> I began trying to use dynamic metadata to achieve the result above. Dynamic metadata involves writing extra YAML blocks into your Quarto document which can include programmatically specified values of parameters, which can then be used by including conditional content by <a href="https://quarto.org/docs/authoring/conditional.html#matching-against-metadata" target="_blank" rel="noopener">matching against them</a>. I found that I could achieve what I wanted if I used the <code>--metadata-file</code> argument to <code>quarto render</code> with the settings included in their own YAML files <del>except for modifying the <code>execute</code> state of the Quarto document, it seems that must be specified in the first YAML block/header. And this cannot be controlled by the <code>--execute</code> and <code>--no-execute</code> flags to <code>quarto render</code> because one needs the code which generates the additional YAML blocks to be run</del> (with thanks to Christophe for pointing this out to me).</p> <p>For Quarto documents using the knitr engine, the R package vignette shows how to use the new <code>write_yaml_metadata_block()</code> function within an R code chunk with output type <code>asis</code> to write the YAML block. In case it is useful to anyone, below I show examples of how to write the YAML blocks in each of the three other engines I have been using. (Admittedly this is not needed if one uses the <code>--metadata-file</code> argument to <code>quarto render</code> in which these settings are included in their own YAML files.)</p> <h3 id="python-jupyter-python3">Python (jupyter: python3)</h3> <pre><code class="language-plaintext">```{python} #| include: false #| tags: [parameters] solutions = 'true' ``` ```{python} #| include: false from IPython.display import Markdown ymltxt = f&quot; solutions: {solutions}&quot; if solutions == 'true': titletxt = &quot;title: Solutions document&quot; else: titletxt = &quot;title: Questions document&quot; ``` `{python} Markdown(&quot;---&quot;)` `{python} Markdown(&quot;params:&quot;)` `{python} Markdown(ymltxt)` `{python} Markdown(titletxt)` `{python} Markdown(&quot;---&quot;)` ::: {.content-visible when-meta=&quot;params.solutions&quot;} ```{python} print(&quot;A solution, which is hidden in questions&quot;) ``` ::: </code></pre> <h3 id="stata-jupyter-nbstata">Stata (jupyter: nbstata)</h3> <pre><code class="language-plaintext">```{stata} *| include: false local solutions : env SOLUTIONS_STATA scalar ymltxt = &quot; solutions: `solutions'&quot; if &quot;`solutions'&quot; == &quot;true&quot; { scalar titletxt = &quot;title: Solutions document&quot; } else { scalar titletxt = &quot;title: Questions document&quot; } ``` `{stata} &quot;---&quot;` `{stata} &quot;params:&quot;` `{stata} scalar(ymltxt)` `{stata} scalar(titletxt)` `{stata} &quot;---&quot;` ::: {.content-visible when-meta=&quot;params.solutions&quot;} ```{stata} display &quot;A solution, which is hidden in questions&quot; ``` ::: </code></pre> <h3 id="julia-engine-julia">Julia (engine: julia)</h3> <pre><code class="language-plaintext">```{julia} #| tags: [parameters] ``` ```{julia} #| echo: false #| output: asis println(&quot;---&quot;) if solutions println(&quot;title: Solutions document&quot;) ymltxt = &quot; solutions: true&quot; else println(&quot;title: Questions document&quot;) ymltxt = &quot; solutions: false&quot; end println(&quot;params:&quot;) println(ymltxt) println(&quot;---&quot;) ``` ::: {.content-visible when-meta=&quot;params.solutions&quot;} ```{julia} println(&quot;A solution, hidden in questions&quot;) ``` ::: </code></pre> <h2 id="summary">Summary</h2> <p>I have shown how to create Quarto profiles for creating tutorial worksheets; one for the questions and one for the solutions from the same Quarto document; for several Quarto engines (<code>engine: knitr</code>, <code>jupyter: python3</code>, <code>jupyter: nbstata</code>, and <code>engine: julia</code>). I have also shown how additional metadata may be written into your Quarto document in these engines which can be used in conjunction with parameterised documents and conditional content.</p> Investigating running R on RISC-V thanks to r-base on Ubuntu https://remlapmot.github.io/post/2025/riscv-r/ Tue, 10 Jun 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/riscv-r/ <h2 id="introduction">Introduction</h2> <p>I was interested to see whether and how well R would run on the new RISC-V architecture.</p> <p>A while ago I read that RISC-V is now a <a href="https://ubuntu.com/blog/empowering-risc-v-with-open-source-through-ubuntu" target="_blank" rel="noopener">first class architecture for Ubuntu</a>.</p> <p>This got me thinking, instead of having to build R from source maybe the <code>r-base</code> package might be available for RISC-V. It turns out that this is indeed the case, the architecture we are interested in is <code>riscv64</code>. The launchpad page for <code>r-base</code> is <a href="https://launchpad.net/ubuntu/&#43;source/r-base" target="_blank" rel="noopener">here</a>. Clicking through the subpages for each version of Ubuntu I can see that R is available for RISC-V from Ubuntu Focal Fossa onwards (for which the version of R is 3.6.3; and the latest version of Ubuntu has the current version of R of 4.5.0).</p> <img src="https://remlapmot.github.io/post/2025/riscv-r/img/questing-r-base-builds.png" alt="Screenshot of the architectures the r-base package is built for Ubuntu Questing Quokka." width="630" style="display: block; margin: auto;"> <p>Why do this? I have no immediate need for this. However, there are now quite a few affordable RISC-V single board computers available, and so a similar argument holds to that made by <a href="https://r4pi.org/" target="_blank" rel="noopener">the R4Pi</a> (R for the Raspberry Pi project) that running R on such affordable machines is a <a href="https://youtu.be/imYEdQ81JPk?si=B97GQJl846WC6zr3" target="_blank" rel="noopener">great benefit because it opens R up to a whole new user base and whole new set of low power use cases</a>.</p> <h2 id="emulating-risc-v-on-an-apple-silicon-mac">Emulating RISC-V on an Apple Silicon Mac</h2> <p>I don&rsquo;t have a RISC-V computer, therefore, I needed to use emulation.</p> <p>My setup is that I&rsquo;m on an Apple Silicon M4 MacBook Air. I thought this might be promising to use because this has an ARM processor which is a reduced instruction set architecture, as is RISC-V.</p> <p>I wondered whether to try <a href="https://mac.getutm.app/" target="_blank" rel="noopener">UTM</a> or <a href="https://www.qemu.org/" target="_blank" rel="noopener">QEMU</a>. I tried UTM first but I couldn&rsquo;t make any progress. So I found a <a href="https://www.reddit.com/r/RISCV/comments/t19dqz/comment/hyfm8s8/" target="_blank" rel="noopener">tip online</a> saying that RISC-V Ubuntu could be launched under QEMU on Ubuntu Linux using the following command.</p> <pre><code class="language-sh">qemu-system-riscv64 \ -machine virt \ -nographic \ -m 12288 -smp 4 \ -bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf \ -kernel /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf \ -device virtio-net-pci,netdev=eth0 \ -netdev user,id=eth0,hostfwd=tcp::2222-:22 \ -drive file=ubuntu-24.10-preinstalled-server-riscv64.img,format=raw,if=virtio </code></pre> <p>Firstly one needs to obtain the Ubuntu img. Following <a href="https://canonical-ubuntu-boards.readthedocs-hosted.com/en/latest/how-to/qemu-riscv/" target="_blank" rel="noopener">this incredible guide by Canonical</a> we can choose one of the three versions of Ubuntu listed (Noble, Oracular, and Plucky).</p> <p>The image downloads as an <em>.img.xz</em> archive, which you can extract by installing <code>xz</code> (I use <a href="https://brew.sh/" target="_blank" rel="noopener">Homebrew</a> for system packages on macOS)</p> <pre><code class="language-sh">brew install xz </code></pre> <p>and decompressing with</p> <pre><code class="language-sh">xz --decompress ubuntu-24.10-preinstalled-server-riscv64.img.xz </code></pre> <p>The extracted file is about 4GB, but later on I realised I needed a slightly larger harddisk for the virtual machine, so I increased it to 8GB with the following command.</p> <pre><code class="language-sh">qemu-img resize ubuntu-24.10-preinstalled-server-riscv64.img 8G </code></pre> <p>I then realised that I needed QEMU on my Mac. Again Homebrew to the rescue with the following command.</p> <pre><code class="language-sh">brew install qemu u-boot-tools </code></pre> <p>(Admittedly I don&rsquo;t think I ended up using the <em>u-boot-tools</em>.) Next I needed the two files; <em>fw_jump.elf</em> and <em>uboot.elf</em>. I had a look in <em>/opt/homebrew/Cellar/qemu/10.0.2/</em> but I couldn&rsquo;t work out if they are in there or not (there are some zip archives in some subdirectories). There are some official documentation pages <a href="https://risc-v-getting-started-guide.readthedocs.io/en/latest/linux-qemu.html" target="_blank" rel="noopener">here</a> and <a href="https://risc-v-getting-started-guide.readthedocs.io/en/latest/linux-qemu.html" target="_blank" rel="noopener">here</a> but I couldn&rsquo;t follow them. I then found a comment that said you can <a href="https://www.reddit.com/r/RISCV/comments/t19dqz/comment/hyfaeh4/" target="_blank" rel="noopener">copy them from an Ubuntu installation</a>, which I implemented in Docker.</p> <pre><code class="language-sh">docker run -it --rm --platform linux/arm64 \ -v $PWD:/home ubuntu:24.04 bash /home/copy-files.sh </code></pre> <p>where <em>copy-files.sh</em> contains</p> <pre><code class="language-sh">apt update apt upgrade -y apt install -y opensbi qemu-system-misc u-boot-qemu cp /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf /home/uboot.elf cp /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf /home/fw_jump.elf </code></pre> <p>This saves the two files to the current working directory, so I could modify my call to <code>qemu-system-riscv64</code> to be as follows.</p> <pre><code class="language-sh">qemu-system-riscv64 \ -machine virt \ -nographic \ -m 12288 -smp 4 \ -bios fw_jump.elf \ -kernel uboot.elf \ -device virtio-net-pci,netdev=eth0 \ -netdev user,id=eth0,hostfwd=tcp::2222-:22 \ -drive file=ubuntu-24.10-preinstalled-server-riscv64.img,format=raw,if=virtio </code></pre> <p>After a couple of seconds you obtain a GRUB screen in which you select the default of <em>Ubuntu</em>. Then after a further approx. 20 seconds of screen output, a little bit to my surprise this worked and I was presented with the login screen to Ubuntu server. The default username is <em>ubuntu</em> and the default password is <em>ubuntu</em>. On login you are immediately prompted to change the password but then you&rsquo;re in.</p> <img src="https://remlapmot.github.io/post/2025/riscv-r/img/ubuntu-launch-screen.png" alt="Screenshot of the Ubuntu server startup message running on an emulated RISC-V virtual machine." width="630" style="display: block; margin: auto;"> <p>So next it&rsquo;s essentially standard Ubuntu commands to update the system and install <code>r-base</code>. I also install <code>r-base-dev</code> to obtain the necessary compilers to build any source packages containing code which needs to be compiled.</p> <pre><code class="language-bash">sudo apt-get update sudo apt-get upgrade -y sudo apt-get install -y r-base r-base-dev </code></pre> <p>Then we launch R.</p> <img src="https://remlapmot.github.io/post/2025/riscv-r/img/r-startup-message.png" alt="Screenshot of the R startup message running on an emulated RISC-V virtual machine." width="630" style="display: block; margin: auto;"> <p>From this point on everything I tried simply worked. I installed <em>data.table</em> from source. Then slightly more obscurely, I tried out a trick from Jeroen Ooms who said that if an R package only contains R code then its binary version will install under any architecture. I have a few R packages in my r-universe that only contain R code, my example installed as expected (note this was built on x86_64 Ubuntu Linux but contains no source code which needs to be compiled).</p> <pre><code class="language-r">install.packages('tmsens', repos = 'https://remlapmot.r-universe.dev/bin/linux/noble-x86_64/4.5/') #&gt; * installing *binary* package โ€˜tmsensโ€™ ... #&gt; * DONE (tmsens) #&gt; #&gt; The downloaded source packages are in #&gt; โ€˜/tmp/RtmpYqwF8W/downloaded_packagesโ€™ library(tmsens) #&gt; Warning message: #&gt; package โ€˜tmsensโ€™ was built under R version 4.5.0 help(package = 'tmsens') #&gt; #&gt; Information on package โ€˜tmsensโ€™ </code></pre> <p>I admit I haven&rsquo;t tried many features here but I am really impressed with Ubuntu packages being available for RISC-V.</p> <p>Once you are finished using R, exit R as usual and to shutdown the Ubuntu server issue the following.</p> <pre><code class="language-sh">sudo poweroff </code></pre> <h2 id="summary">Summary</h2> <p>I have shown how to install and run R on Ubuntu Server for RISC-V under QEMU emulation. Thanks to Canonical&rsquo;s support for RISC-V and the maintainers of the <code>r-base</code> and related packages the experience of running R on RISC-V is already excellent.</p> How to widen the accordion on a Blackboard page https://remlapmot.github.io/post/2025/blackboard-accordion/ Fri, 30 May 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/blackboard-accordion/ <h2 id="introduction">Introduction</h2> <p>My university uses the Blackboard online learning environment (there are others, e.g., Moodle and Canvas, etc.). On several of the Unit (aka Module) and short course sites we have an accordion containing the table with the timetable of the teaching. Unfortunately I find the default width too narrow.</p> <img src="https://remlapmot.github.io/post/2025/blackboard-accordion/img/accordion-expanded.png" alt="Screenshot of expanded accordion." width="630" style="display: block; margin: auto;"> <p>This post shows how to make this accordion element wider.</p> <h2 id="widening-the-accordion">Widening the accordion</h2> <p>First we need to work out what type of element it is. To do this in Google Chrome right click on a part of the accordion and click <em>Inspect</em>.</p> <img src="https://remlapmot.github.io/post/2025/blackboard-accordion/img/devtools-inspect.png" alt="Screenshot of expanded accordion." width="630" style="display: block; margin: auto;"> <p>We can see that the accordion is a div of class deo-accordion. And we can see that it&rsquo;s <code>max-width</code> is currently set to 40em which is 640 pixels.</p> <p>Therefore to modify this we need to write the CSS for this with a larger <code>max-width</code>. It turns out Blackboard does not allow us to put CSS in a <code>&lt;style&gt;</code> tag directly in a component&rsquo;s source code. Therefore, we put the following in a file called <em>mystyles.css</em>. The 60em is equivalent to 960 pixels.</p> <pre><code class="language-css">.deo-accordion { max-width: 60em; } </code></pre> <p>We then upload this file to our Blackboard site&rsquo;s <em>Content</em> area. Once uploaded we obtain the <em>Permanent URL</em> of the file by clicking the dropdown arrow to the right of the filename and selecting <em>360ยฐ View</em>. Copy the permanent URL to the clipboard.</p> <img src="https://remlapmot.github.io/post/2025/blackboard-accordion/img/360-degree-view.png" alt="Screenshot of 360ยฐ View of our CSS file." width="630" style="display: block; margin: auto;"> <p>Within the content of the item on your Blackboard page which contains the accordion, click edit, then click the <code>&lt;&gt;</code> symbol to enter editing source code mode. At the bottom of the code enter the following line &ndash; replacing the <code>PERMANENT_URL</code> with what we just copied and save the changes.</p> <pre><code class="language-html">&lt;link href=&quot;PERMANENT_URL&quot; rel=&quot;stylesheet&quot;&gt; </code></pre> <p>And you should find that you have a wider accordion.</p> <img src="https://remlapmot.github.io/post/2025/blackboard-accordion/img/wider-accordion.png" alt="Screenshot of 360ยฐ View of our CSS file." width="630" style="display: block; margin: auto;"> <h2 id="summary">Summary</h2> <p>I have shown how to widen the accordion on a Blackboard page with some simple CSS.</p> Some more R, Git, and R Markdown and Quarto tips https://remlapmot.github.io/talk/2025_code-clinic/ Wed, 07 May 2025 12:00:00 +0000 https://remlapmot.github.io/talk/2025_code-clinic/ Running the nbstata Jupyter kernel within a uv virtual environment https://remlapmot.github.io/post/2025/nbstata-uv-venv/ Tue, 29 Apr 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/nbstata-uv-venv/ <h2 id="introduction">Introduction</h2> <p>Any new project using Python or Jupyter is very strongly recommended to use a virtual environment. A virtual environment is a directory (usually at the top level; often called either <em>.venv</em> or <em>venv</em>) within your project directory which contains the dependency Python packages and perhaps the Python installation (or pointers to the Python installation on your system).</p> <p>There are many Python project managers, in this post I will use the new and extremely fast project manager <a href="https://docs.astral.sh/uv/" target="_blank" rel="noopener">uv</a>. I will show how to use the <a href="https://hugetim.github.io/nbstata/" target="_blank" rel="noopener">nbstata</a> Jupyter kernel by Tim Huegerich within a uv virtual environment on macOS, Windows, and Linux. The aim is to be able to conveniently and reproducibly render Quarto documents using the <a href="https://hugetim.github.io/nbstata/user_guide.html#quarto-tips" target="_blank" rel="noopener"><code>jupyter: nbstata</code></a> engine from within the virtual environment.</p> <p>At this point it&rsquo;s worth saying that I am a contributor to the excellent <a href="https://cran.r-project.org/package=Statamarkdown" target="_blank" rel="noopener">Statamarkdown</a> R package by Doug Hemken. This can be used in Quarto documents using the knitr engine. In this post I&rsquo;ll be using nbstata because nbstata uses <a href="https://www.stata.com/python/pystata/" target="_blank" rel="noopener">pystata</a> which is StataCorp&rsquo;s official Python package (included in each Stata installation) and is their official way of integrating Stata in Python. As such pystata and hence nbstata have more features than Statamarkdown can provide.</p> <p>And it is worth emphasizing how useful virtual environments are because it is very easy to get oneself in a complete mess with regards Python, as you will likely end up with many different versions installed. Chaos can then accidentally ensue regarding which versions of dependency packages your different projects require given a certain Python version, as always XKCD have a comic illustrating the problem.</p> <figure> <img src="https://imgs.xkcd.com/comics/python_environment.png" alt="XKCD 1987, a diagram showing the chaos of not using Python virtual environments." width="630" style="display: block; margin: auto;"> <figcaption>XKCD 1987, Python Environment. <a href="https://xkcd.com/license.html">XKCD license</a></figcaption> </figure> <h2 id="setup-basics">Setup basics</h2> <p>On your machine you need to have</p> <ul> <li>Stata installed and know the location of the installation <ul> <li>and check that within the installation there is the <em>utilities</em> directory containing the <em>pystata</em> directory</li> </ul> </li> <li>uv must be installed, see <a href="https://docs.astral.sh/uv/getting-started/installation/" target="_blank" rel="noopener">its docs for installation instructions</a></li> <li>to render Quarto documents you need <a href="https://quarto.org/docs/get-started/" target="_blank" rel="noopener">Quarto installed</a></li> <li>and of course a text editor and a terminal.</li> <li>On Windows I would recommend using Git Bash as your shell (this is bundled with <a href="https://git-scm.com/downloads/win" target="_blank" rel="noopener">Git for Windows</a>) for running the setup script in the next section.</li> </ul> <h2 id="a-shell-script-to-setup-the-uv-virtual-environment">A shell script to setup the uv virtual environment</h2> <p>What follows are the shell commands we need to run to setup our virtual environment. The script is for macOS but I have included comments in the places where amendments are required for either Windows or Linux. I have also included this script <a href="https://github.com/remlapmot/nbstata-uv-setup" target="_blank" rel="noopener">in a repository</a> (in fact I have included a script per operating system).</p> <p>Note that currently on macOS and Linux there cannot be any spaces in the filepath to your virtual environment. And you might need to amend some of the paths, say if you have a different version of Stata or if it&rsquo;s installation location is different.</p> <ul> <li>On Linux we need to ensure that the directory containing our stata/stata-mp binary is on <code>PATH</code>.</li> </ul> <pre><code class="language-bash"># export PATH=$PATH:/usr/local/stata18 </code></pre> <ul> <li>We then need to ensure that our Python installation will be able to find the pystata package. We can do this by defining the <code>PYTHONPATH</code> environment variable.</li> </ul> <pre><code class="language-bash"># Required so Python can find pystata package export PYTHONPATH=/Applications/Stata/utilities # Linux: export PYTHONPATH=/usr/local/stata18/utilities # Windows: export PYTHONPATH=&quot;C:/Program Files/Stata18/utilities&quot; </code></pre> <ul> <li>We then setup the virtual environment. This will create a directory in our project called <em>.venv</em>. I prefer to explicitly set the version of Python I am using.</li> </ul> <pre><code class="language-bash">uv venv --python 3.13 </code></pre> <ul> <li>We then activate the virtual environment</li> </ul> <pre><code class="language-bash">source .venv/bin/activate # Windows: source .venv/Scripts/activate </code></pre> <ul> <li>Now we can install the required Python dependency packages as per the <a href="https://hugetim.github.io/nbstata/user_guide.html#install-nbstata" target="_blank" rel="noopener">nbstata docs</a>.</li> </ul> <pre><code class="language-bash"># Install nbstata dependency Python packages uv pip install jupyterlab nbstata jupyterlab_stata_highlight2 # Additional Python packages if using parameterised Quarto documents # uv pip install papermill python-dotenv jupyter-cache </code></pre> <ul> <li>Then we setup the nbstata Jupyter kernel</li> </ul> <pre><code class="language-bash">python -m nbstata.install </code></pre> <p>The nbstata docs states that the <code>--sys-prefix</code> flag may be required in a virtual environment but I haven&rsquo;t found that to be the case.</p> <ul> <li>Then we run commands to do our actual work; for example rendering a Quarto document using the <code>jupyter: nbstata</code> engine. An example Quarto document, say <em>index.qmd</em>, could be:</li> </ul> <pre><code class="language-plaintext">--- title: My example nbstata document jupyter: nbstata --- ```{stata} display &quot;Hello, World!&quot; ``` </code></pre> <pre><code class="language-bash">quarto render index.qmd </code></pre> <p>Note that Quarto documents using the nbstata kernel can also be embedded within other Quarto documents, <a href="https://remlapmot.github.io/post/2025/multi-engine-quarto/" target="_blank" rel="noopener">as I have described in a previous post</a>.</p> <ul> <li>And that&rsquo;s it. When you have finished your work you can deactivate the virtual environment</li> </ul> <pre><code class="language-bash">deactivate </code></pre> <h2 id="recording-this-as-a-project-with-pyprojecttoml-and-uvlock-files">Recording this as a project with pyproject.toml and uv.lock files</h2> <p>And of course we can go a step further by setting this up as a proper Python project. This will have a <em>pyproject.toml</em> file to record the dependencies and a <em>uv.lock</em> file to record their exact versions. To do this run</p> <pre><code class="language-bash">uv init </code></pre> <p>You can delete the <em>main.py</em> file that&rsquo;s created here, but commit everything else into your repo. Then add the dependencies with</p> <pre><code class="language-bash">uv add jupyterlab nbstata jupyterlab_stata_highlight2 </code></pre> <p>You will see these are entered into the <em>pyproject.toml</em> and <em>uv.lock</em> files. The <em>pyproject.toml</em> file will now look something like this (I have edited the <code>description</code>)</p> <pre><code class="language-toml">[project] name = &quot;nbstata-uv-setup&quot; version = &quot;0.1.0&quot; description = &quot;This project demonstrates how to use the nbstata Jupyter kernel in a uv virtual environment, so that one can render Quarto documents using this engine.&quot; readme = &quot;README.md&quot; requires-python = &quot;&gt;=3.13&quot; dependencies = [ &quot;jupyterlab&gt;=4.4.2&quot;, &quot;jupyterlab-stata-highlight2&gt;=0.1.2&quot;, &quot;nbstata&gt;=0.8.2&quot;, ] </code></pre> <p>Commit these files into your repo.</p> <p>At a later date you can restore this environment by running</p> <pre><code class="language-bash">uv sync # which creates the .venv # Then activate the virtual environment, etc source .venv/bin/activate # Windows: source .venv/Scripts/activate </code></pre> <p>You find out more about uv project commands in its <a href="https://docs.astral.sh/uv/guides/projects/" target="_blank" rel="noopener">documentation</a>.</p> <h2 id="summary">Summary</h2> <p>I have shown how to setup and run the nbstata Jupyter kernel within a uv virtual environment on macOS, Windows, and Linux; in order to conveniently and reproducibly render Quarto documents using the <code>jupyter: nbstata</code> engine. And I have also shown how to set this up as a Python project.</p> Running R on Windows on ARM on GitHub Actions https://remlapmot.github.io/post/2025/win-11-arm-gha/ Wed, 16 Apr 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/win-11-arm-gha/ <h2 id="introduction">Introduction</h2> <p>GitHub has recently announced that Windows ARM64 runners are <a href="https://github.blog/changelog/2025-04-14-windows-arm64-hosted-runners-now-available-in-public-preview/" target="_blank" rel="noopener">now available</a> under the <code>windows-11-arm</code> label.</p> <p>I help maintain an R package, <a href="https://mrcieu.github.io/TwoSampleMR/" target="_blank" rel="noopener">TwoSampleMR</a>, which has quite alot of users. The package is not on CRAN because several of its dependencies are only on GitHub, and for a package to be on CRAN essentially all of its dependencies must also be on CRAN. As a result I am always interested to try installing the package on new operating systems and architectures.</p> <p>(In this post I will use ARM and AARCH64 interchangeably.)</p> <h2 id="setting-up-r-aarch64-on-windows-on-arm">Setting up R AARCH64 on Windows on ARM</h2> <h3 id="avoiding-confusion-with-the-default-runner-software">Avoiding confusion with the default runner software</h3> <p>It is important to mention that the x86_64 version of R 4.4.2 and RTools44 are included in the <a href="https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md" target="_blank" rel="noopener">default software set</a> for the <code>windows-latest</code> GitHub Actions runner. And the directory including its binaries are on the <code>PATH</code> environment variable (specifically <em>C:\Program Files (x86)\R\R-4.4.2\bin\x64</em>). As a result if you run <code>R</code>, <code>Rscript</code>, or <code>R CMD batch</code> etc. in a shell in the runner you will obtain the x86_64 version of R (which runs under emulation on the ARM runner). Let&rsquo;s say this is not what we want, so to setup the ARM version of R we need to install it ourselves.</p> <h3 id="installing-aarch64-r-and-rtools45">Installing AARCH64 R and RTools45</h3> <p>Tomas Kalibera from the R Core Team has provided several excellent posts ( <a href="https://blog.r-project.org/2023/08/23/will-r-work-on-64-bit-arm-windows/index.html" target="_blank" rel="noopener">here</a> and <a href="https://blog.r-project.org/2024/04/23/r-on-64-bit-arm-windows/index.html" target="_blank" rel="noopener">here</a>) about R for Windows on ARM, and installers for it have been available for some time.</p> <p>The r-hub API does not yet provide the installer information for the AARCH64 version of R, so I came up with the following workflow file - amended from r-lib/actions to install R 4.5.0 and RTools45. Place such a (GitHub Actions workflow) file in a public GitHub repo in a <em>.github/workflows</em> directory, and enable GitHub Actions in the repo settings.</p> <pre><code class="language-yaml">on: push: branches: [main, master] pull_request: branches: [main, master] workflow_dispatch: name: Check-install-win-11-arm permissions: read-all jobs: windows-11-on-arm: runs-on: windows-11-arm name: windows-11-arm strategy: fail-fast: false env: GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} R_KEEP_PKG_SOURCE: yes steps: - name: Install R and RTools for Windows on ARM and install TwoSampleMR run: | $url = &quot;https://www.r-project.org/nosvn/winutf8/aarch64/R-4-signed/R-4.5.0-aarch64.exe&quot; Invoke-WebRequest -Uri &quot;$url&quot; -OutFile R-4.5.0-aarch64.exe -UseBasicParsing -UserAgent &quot;NativeHost&quot; Start-Process -FilePath R-4.5.0-aarch64.exe -ArgumentList &quot;/install /norestart /verysilent /SUPPRESSMSGBOXES&quot; -NoNewWindow -Wait $url = &quot;https://cran.r-project.org/bin/windows/Rtools/rtools45/files/rtools45-aarch64-6536-6492.exe&quot; Invoke-WebRequest -Uri &quot;$url&quot; -OutFile rtools45-aarch64-6536-6492.exe -UseBasicParsing -UserAgent &quot;NativeHost&quot; Start-Process -FilePath rtools45-aarch64-6536-6492.exe -ArgumentList &quot;/install /norestart /verysilent /SUPPRESSMSGBOXES&quot; -NoNewWindow -Wait $rscript = &quot;C:\Program Files\R-aarch64\R-4.5.0\bin\Rscript.exe&quot; $arguments = &quot;-e&quot;, &quot;print(R.version); # the rest of your R code goes here ...&quot; &amp; $rscript $arguments </code></pre> <p>Breaking down the final <code>steps</code> section of this;</p> <ul> <li>we define the url of the R 4.5.0 aarch64 installer;</li> <li>we then download the installer using <code>Invoke-WebRequest</code> (note that the default shell in Windows is Powershell);</li> <li>we then run the installer using <code>Start-Process</code>. I am not sure if I need all of the arguments I have specified here but it seems to work.</li> <li>We then do the same for RTools45.</li> <li>We then define a variable for the path to the <em>Rscript.exe</em> binary;</li> <li>we define a variable containing the arguments we want to pass to <em>Rscript</em>;</li> <li>we then invoke <em>Rscript</em> using our two variables and the <code>&amp;</code> call operator.</li> </ul> <p>Then we navigate to our GitHub repo and view the output in the Actions tab under the relevant run.</p> <p>Of course if you want to run your own R script you&rsquo;ll need an initial step to checkout your repo.</p> <p>To confirm that we really have launched the AARCH64 version of R we see the output of <code>print(R.version)</code> is as follows.</p> <pre><code class="language-r">print(R.version) #&gt; _ #&gt; platform aarch64-w64-mingw32 #&gt; arch aarch64 #&gt; os mingw32 #&gt; crt ucrt #&gt; system aarch64, mingw32 #&gt; status #&gt; major 4 #&gt; minor 5.0 #&gt; year 2025 #&gt; month 04 #&gt; day 11 #&gt; svn rev 88135 #&gt; language R #&gt; version.string R version 4.5.0 (2025-04-11 ucrt) #&gt; nickname How About a Twenty-Six </code></pre> <h2 id="summary">Summary</h2> <p>I have shown how to install the AARCH64 version of R and RTools45 on the recently released Windows on ARM runner in GitHub Actions.</p> <p>As an aside, I note that we are now in the interesting position in that GitHub Actions now has Windows, macOS, and Ubuntu Linux all available on both x86_64 and ARM architectures.</p> Amending the Git commit message of a previous commit (that isn't the most recent) in GitHub Desktop without performing an interactive rebase https://remlapmot.github.io/post/2025/amend-commit-messages/ Sat, 08 Mar 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/amend-commit-messages/ <h2 id="introduction">Introduction</h2> <p>As R developers I think we can all agree that Git is hard. There won&rsquo;t be many of us who at some time haven&rsquo;t broken a Git repository in some way or other or lost some work, I know that I have (several times &hellip; ahem).</p> <p>A task I sometimes need to achieve when working on a branch is amending a commit message. I use <a href="https://github.com/apps/desktop" target="_blank" rel="noopener">GitHub Desktop</a> to help me with Git, and I recommend it to all my students. If the commit you want to amend the message of is the most recent commit you can simply right click on it and select <em>Amend Commit&hellip;</em>.</p> <img src="https://remlapmot.github.io/post/2025/amend-commit-messages/img/01-right-click-amend-commit.png" alt="Screenshot of amending a commit in GitHub Desktop." width="630" style="display: block; margin: auto;"> <p>This is providing a user friendly interface to running</p> <pre><code class="language-sh">git commit --amend </code></pre> <p>in the terminal for us. This is all covered in the <a href="https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/changing-a-commit-message" target="_blank" rel="noopener">GitHub documentation</a>.</p> <p>However, what if the commit is not the most recent &ndash; when you right click those commits the option to amend them is not available. If your commits after your target commit don&rsquo;t touch the same lines in the same file/s you could reorder your commits such that your target commit is the most recent and then right click and <em>Amend Commit&hellip;</em> again.</p> <p>What if you can&rsquo;t easily or don&rsquo;t want to reorder your commits? At this point you might start questioning your life choices; how did I end up using the same tools as the Linux kernel developers? All I want to do is amend a commit message! Anyway, the proper answer is to perform an interactive rebase, but that is relatively advanced and not without stress. Luckily I have a simple trick in GitHub Desktop to achieve our goal without performing an interactive rebase.</p> <h2 id="the-trick-squashing-an-empty-commit-onto-the-target-commit">The trick: squashing an empty commit onto the target commit</h2> <p>GitHub Desktop allows us to squash to commits together. When it does this it allows us to amend the commit message of the resulting commit. Therefore, to achieve our goal of amending previous commit messages we need to:</p> <ul> <li>Identify the commit you want to amend the message of. Here I have made a typo and want to fix the message to say <em>Use test-rcpp.R</em></li> </ul> <img src="https://remlapmot.github.io/post/2025/amend-commit-messages/img/02-commit-to-amend.png" alt="Screenshot of squashing commits GitHub Desktop." width="630" style="display: block; margin: auto;"> <ul> <li> <p>Create an empty commit</p> <p>For this you will need command line <a href="https://git-scm.com/" target="_blank" rel="noopener">Git</a> installed (GitHub Desktop has a version of Git bundled within it, so not everyone who has GitHub Desktop installed has Git installed separately). Run the following (you don&rsquo;t have to include the message).</p> <pre><code class="language-sh">git commit --allow-empty -m &quot;Empty commit for purposes of trick&quot; </code></pre> </li> <li> <p>Perform a squash in GitHub Desktop by dragging and dropping the empty commit onto your target commit as per the screenshot below (and in the screenshot at the <a href="#top">top</a> of this post). The empty commit has no content, so it does not affect the content of your target commit.</p> </li> </ul> <img src="https://remlapmot.github.io/post/2025/amend-commit-messages/img/03-drag-and-drop.png" alt="Screenshot of dragging the empty commit onto the target commit in GitHub Desktop." width="630" style="display: block; margin: auto;"> <ul> <li>Enter your amended commit message in the <em>Summary</em> box and delete the text in the <em>Description</em> box.</li> </ul> <img src="https://remlapmot.github.io/post/2025/amend-commit-messages/img/06-squashing-commits.png" alt="Screenshot of squashing commits GitHub Desktop." width="630" style="display: block; margin: auto;"> <ul> <li>Click <em>Squash 2 Commits</em>.</li> </ul> <img src="https://remlapmot.github.io/post/2025/amend-commit-messages/img/07-new-commit-message.png" alt="Screenshot of finalising squashed commit in GitHub Desktop." width="630" style="display: block; margin: auto;"> <ul> <li>That&rsquo;s it, we&rsquo;re finished! You can now push your branch upto GitHub (or in my case in the screenshot force push because I had previously pushed this branch to the remote).</li> </ul> <img src="https://remlapmot.github.io/post/2025/amend-commit-messages/img/08-final-state.png" alt="Screenshot of your amend Git history ready to the pushed to GitHub in GitHub Desktop." width="630" style="display: block; margin: auto;"> <h2 id="the-proper-method-performing-an-interactive-rebase">The proper method: performing an interactive rebase</h2> <p>If you want to do achieve this the proper way or amend the contents of previous commits you&rsquo;ll need to perform an interactive rebase. That is a little bit tricky to perform in the terminal, although there are lots of helpful YouTube videos and blogposts showing how to do it.</p> <p>If you ever need to do this I recommend using the <a href="https://github.com/jesseduffield/lazygit" target="_blank" rel="noopener">Lazygit</a> terminal user interface, which has the best interface to interactive rebasing I&rsquo;ve seen. To start the process, navigate to the <em>Reflog</em> pane (by pressing <kbd>Tab</kbd> twice), then use your up and down arrows to select your target commit, and either press <kbd>r</kbd> to reword (i.e., amend the commit message) or <kbd>e</kbd> to edit the commit itself (or choose one of the other options listed at the bottom).</p> <img src="https://remlapmot.github.io/post/2025/amend-commit-messages/img/09-lazygit.png" alt="Screenshot of starting to amend a commit message in the Lazygit TUI." width="630" style="display: block; margin: auto;"> <h2 id="summary">Summary</h2> <p>I have shown how to amend commit messages for commits that aren&rsquo;t the most recent commit in GitHub Desktop without performing an interactive rebase.</p> Checking your R packages and practicals on a schedule using GitHub Actions https://remlapmot.github.io/post/2025/checking-packages-and-practicals/ Mon, 24 Feb 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/checking-packages-and-practicals/ <h2 id="introduction">Introduction</h2> <p>Do you have a R package that&rsquo;s just on GitHub? How often do you check it? CRAN follows a rolling release model, so any day one of your package&rsquo;s dependency packages could be updated - breaking your package!</p> <p>Or maybe you teach a course that runs once a year and it has some R practical sessions? It can be very frustrating when you rerun your practical after a year to find that several of the R packages it uses have been updated, and now you have to work out how to fix things. For this situation we might use <a href="https://rstudio.github.io/renv/index.html" target="_blank" rel="noopener"><strong>renv</strong></a> to record the packages and their versions. But then your course participants will need to use <strong>renv</strong> which might lead to a room full of 30 students all having <strong>renv</strong> problems. Alternatively, you might install all your packages from a single date, meaning you can restore them using the a snapshot date from the <a href="https://packagemanager.posit.co/client/#/repos/cran/setup" target="_blank" rel="noopener">Public Posit Package Manager</a> but you might find your course participants wondering why you use packages that are a year out of date. Another superb solution is to provide course participants with an R environment you have defined and tested your practical works in. Such a solution is offered by creating the practical as a project in a <a href="https://posit.cloud/" target="_blank" rel="noopener">Posit Cloud</a> workspace, indeed Posit Cloud is so good it almost makes running R practicals boring.</p> <p>My solution to these problems is to regularly run <code>R CMD check</code> on your package/run your practical with the latest versions of the required packages throughout the year. This way you&rsquo;ll hopefully pick up any changes in dependency packages long before the next running of your course (although you might still be unlucky).</p> <p>If I had a server I could suggest setting up a <a href="https://linux.die.net/man/8/cron" target="_blank" rel="noopener"><code>cron</code></a> job to check the package or practical. However, most of us don&rsquo;t have a server. Luckily if your package or practical is in a public GitHub repository then we can run GitHub Actions on it. It turns out GitHub Actions has a <a href="https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/events-that-trigger-workflows#schedule" target="_blank" rel="noopener">scheduling facility that uses <code>cron</code> syntax</a>.</p> <p>Next I&rsquo;ll show how to run R code on a schedule on GitHub Actions for these two cases.</p> <h2 id="checking-packages-on-a-schedule">Checking packages on a schedule</h2> <p>I use the actions and example workflows from the <a href="https://github.com/r-lib/actions" target="_blank" rel="noopener">r-lib/actions</a> repository. For checking a package there are several examples, in the <em>examples</em> directory of that repository, that we can copy then amend. Let&rsquo;s use the <a href="https://github.com/r-lib/actions/blob/v2-branch/examples/check-full.yaml" target="_blank" rel="noopener">check-full.yaml</a> example.</p> <p>Copy this file into a <em>.github/workflows</em> directory in the repository for your package. Now we need to enable scheduled running. Amend the first <code>on:</code> block to include the two lines I have inserted below.</p> <pre><code class="language-yaml">on: ... schedule: - cron: &quot;00 9 * * TUE&quot; </code></pre> <p>What does this syntax mean? Luckily there are many websites which will decipher that for us, such as <a href="https://crontab.guru/" target="_blank" rel="noopener">crontab guru</a>. (Nb. you can see a full version of this file in <a href="https://github.com/remlapmot/OneSampleMR/blob/main/.github/workflows/R-CMD-check.yaml" target="_blank" rel="noopener">one of my repositories</a>.)</p> <img src="https://remlapmot.github.io/post/2025/checking-packages-and-practicals/img/cron-tab-guru-screenshot.png" alt="Screenshot from the crontab guru website." width="630" style="display: block; margin: auto;"> <p>From the crontab guru screenshot above, we see this means run at 9:00 am (UTC) on Tuesdays. Once you have committed this file to your repo and pushed it to GitHub on your main/master branch then your automated checking is taken care of. If a check fails GitHub will send a notification and you can then investigate further.</p> <h2 id="running-practicals-on-a-schedule">Running practicals on a schedule</h2> <p>Let&rsquo;s say we have prepared a practical worksheet in a Quarto document. In this case we&rsquo;ll additionally use the actions and examples in the <a href="https://github.com/quarto-dev/quarto-actions" target="_blank" rel="noopener">quarto-dev/quarto-actions</a> repository.</p> <p>The workflow file I ended up with is shown below and is in this <a href="https://github.com/remlapmot/example-exercise-to-run-on-schedule" target="_blank" rel="noopener">example repository</a>. It is essentially an amended version of the <a href="https://github.com/r-lib/actions/blob/v2-branch/examples/render-rmarkdown.yaml" target="_blank" rel="noopener">R Markdown rendering example in r-lib/actions</a>.</p> <pre><code class="language-yaml">on: push: branches: - main workflow_dispatch: schedule: - cron: &quot;0 0 1 * *&quot; # run on 1st day of month name: Render permissions: contents: write jobs: build-deploy: runs-on: ${{ matrix.config.os }} name: ${{ matrix.config.os }} strategy: fail-fast: false max-parallel: 1 matrix: config: - { os: macos-latest } - { os: windows-latest } - { os: ubuntu-latest } env: GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }} steps: - name: Check out repository uses: actions/checkout@v4 - name: Setup R uses: r-lib/actions/setup-r@v2 - name: Set up Quarto uses: quarto-dev/quarto-actions/setup@v2 - name: Setup R packages uses: r-lib/actions/setup-r-dependencies@v2 with: upgrade: 'TRUE' - name: Render practical and commit output documents into repo shell: bash run: | # Render questions Rscript -e 'quarto::quarto_render(&quot;exercise-01.qmd&quot;, output_file = &quot;exercise-01-questions-${{ matrix.config.os }}.html&quot;, execute_params = list(solutions = FALSE, title = &quot;Example exercise: Questions&quot;))' # Render solutions Rscript -e 'quarto::quarto_render(&quot;exercise-01.qmd&quot;, output_file = &quot;exercise-01-solutions-${{ matrix.config.os }}.html&quot;)' # Commit output documents git config --local user.name $GITHUB_ACTOR git config --local user.email [email protected] git pull git add &quot;*.html&quot; git commit * -m &quot;Render practical on ${{ matrix.config.os }}&quot; || echo 'No changes to commit' git push origin || echo 'No changes to commit' </code></pre> <p>Crikey, that&rsquo;s alot, let&rsquo;s break that down.</p> <ul> <li>The <code>on:</code> section defines this will run <ul> <li>on pushes to the main branch,</li> <li>if the workflow dispatch button is clicked in the GitHub interface,</li> <li>and once per month (at midnight on the first day on the month) as defined by our <code>cron</code> syntax</li> </ul> </li> <li>The <code>jobs:</code> section specifies <ul> <li>we will run this on 3 different operating systems, Windows (most of my students/course participants have Windows laptops, followed by macOS), macOS, and Ubuntu Linux. <a href="https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners" target="_blank" rel="noopener">More info about GitHub Actions runners is available</a>;</li> <li><code>max-parallel: 1</code> says that each job will run in turn (this isn&rsquo;t really needed but just to be safe since I will commit the output documents back into the repository)</li> <li><code>env:</code> specifies that we grant the jobs permission with out GitHub token</li> <li><code>steps:</code> defines what will run, which is <ul> <li> <p>we checkout the repo</p> </li> <li> <p>we then install R and Quarto using the relevant actions</p> </li> <li> <p>we setup the R dependency packages. I am not using <strong>renv</strong>, so the action knows how to do this because I have included a <em>DESCRIPTION</em> file in the repo (to the action we are faking that the repo is an R package - this is a trick from Hadley Wickham). The <a href="https://github.com/remlapmot/example-exercise-to-run-on-schedule/blob/main/DESCRIPTION" target="_blank" rel="noopener">full file is in the repository</a> but the key entry is the <em>Imports</em> list of hard depdenency packages.</p> <pre><code class="language-plaintext">... Imports: knitr, quarto, sessioninfo ... </code></pre> </li> </ul> </li> <li>we specify <code>upgrade: 'TRUE'</code> to always install the latest version of the dependency packages.</li> <li>then we finally render the two versions of our Quarto document. Here we append the operating system name into the output document filenames and we commit these files back into the repository for our records. We have to specify <code>shell: bash</code> because otherwise the Windows runner will use Powershell and the environment variable syntax in the <code>git config</code> commands would be incorrect. Also we need a <code>git pull</code> before making the commit because we have 3 jobs running in series, so there will be new/amended files in the repo after the first and second jobs.</li> </ul> </li> </ul> <p>Phew!</p> <p>It&rsquo;s worth pointing out that instead of a Quarto document our practical could be in an R Markdown document or an R script (we&rsquo;d just need to make the relevant changes in the workflow file above such as adding Pandoc and amending the render commands).</p> <p>Note, GitHub Actions only works for free in a public repository; so obviously this can&rsquo;t be used for any material that is assessed.</p> <h2 id="surprise-benefits">Surprise benefits</h2> <p>A nice side effect of using the <code>r-lib/actions/setup-r</code> and <code>quarto-dev/quarto-actions/setup</code> actions is that they update to the release version of R/Quarto as updates are released. So you don&rsquo;t have to worry about updating the version of R/Quarto.</p> <h2 id="github-actions-woes">GitHub Actions woes</h2> <p>At this point I acknowledge that sometimes GitHub Actions can be more trouble than they are worth. This is because they often fail for reasons which are not problems with your code. For example, I had one repository in which the Windows runner would regularly lose internet connection (I have no idea why) half way through installing the R dependency packages and hence would regularly notify me of a failed check. Also you need to keep the code in your workflow file up-to-date. Usually a quick comparison to the current version of the example in the r-lib/actions repo is enough to show you what you need to change.</p> <h2 id="summary">Summary</h2> <p>I have shown how to use GitHub Actions scheduling feature to automatically run scheduled checks on R packages and R scripts/R Markdown/Quarto documents for practicals enabling you to keep on top of any changes in your dependency packages and indeed in any changes in the release versions of R and Quarto.</p> Creating a Finder Smart Folder of your RStudio Project files to enable super fast project launching https://remlapmot.github.io/post/2025/smart-folders/ Fri, 21 Feb 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/smart-folders/ <h2 id="introduction">Introduction</h2> <p>When I&rsquo;m switching projects I naturally start with Finder. I&rsquo;m sure if I became a Terminal warrior there are faster ways of searching for files (e.g. the <a href="https://github.com/alexpasmantier/television" target="_blank" rel="noopener">Television</a> fuzzy finder TUI). And I know there are other ways to avoid using Finder, e.g., using Spotlight, <a href="https://www.alfredapp.com/" target="_blank" rel="noopener">Alfred</a>, or <a href="https://www.raycast.com/" target="_blank" rel="noopener">Raycast</a>; and indeed I do have <a href="https://rstats.wtf/projects#tricks-for-opening-projects" target="_blank" rel="noopener">this fantastic tip</a> that allows Alfred to find RStudio project files setup. But something to do with how I learnt to use a computer just naturally means I&rsquo;m wedded to Finder, but navigating through my mess of nested directories wastes time and energy.</p> <p>I realised that Finder on macOS has a helpful feature called Smart Folders. We can use this to setup a saved search of RStudio Project <em>.Rproj</em> files (and/or VSCode/Positron <em>.code-workspace</em> files) to allow us to see all the <em>.Rproj</em> files on our computer. This makes finding and opening projects from within Finder fast and convenient.</p> <h2 id="setting-up-a-smart-folder-of-your-rstudio-project-files">Setting up a Smart Folder of your RStudio project files</h2> <p>In Finder;</p> <ul> <li>navigate to the folder you want your search to start in, either your home folder or your Documents folder are obvious candidates</li> <li>click File | New Smart Folder</li> </ul> <img src="https://remlapmot.github.io/post/2025/smart-folders/img/new-smart-folder.png" alt="Screenshot of creating a new Smart Folder in macOS Finder." width="630" style="display: block; margin: auto;"> <ul> <li>near the top right corner click the plus button next to the <em>Save</em> button.</li> </ul> <img src="https://remlapmot.github.io/post/2025/smart-folders/img/plus-to-right-of-save.png" alt="Screenshot of creating a new condition as part for a new Smart Folder in macOS Finder." width="630" style="display: block; margin: auto;"> <ul> <li>we need to add the <em>File extension</em> attribute to the choices, click <em>Name</em> near the top left, then <em>Other&hellip;</em></li> </ul> <img src="https://remlapmot.github.io/post/2025/smart-folders/img/name-other.png" alt="Screenshot of adding a new attribute as part for a new Smart Folder in macOS Finder." width="630" style="display: block; margin: auto;"> <ul> <li>Then search for <em>File extension</em> and check the box on the right handside</li> </ul> <img src="https://remlapmot.github.io/post/2025/smart-folders/img/adding-file-extension-attribute.png" alt="Screenshot of adding the file extension attribute as part for a new Smart Folder in macOS Finder." width="630" style="display: block; margin: auto;"> <ul> <li>enter <em>Rproj</em> in the box (it doesn&rsquo;t seem to matter if you include the . first)</li> </ul> <img src="https://remlapmot.github.io/post/2025/smart-folders/img/entering-file-extension.png" alt="Screenshot of adding the file extension condition as part for a new Smart Folder in macOS Finder." width="630" style="display: block; margin: auto;"> <ul> <li>click <em>Save</em> (near top right corner) and give it a sensible name, e.g., <em>Rproj.savedSearch</em></li> </ul> <img src="https://remlapmot.github.io/post/2025/smart-folders/img/saving.png" alt="Screenshot of saving the new Smart Folder in macOS Finder." width="630" style="display: block; margin: auto;"> <p>And you&rsquo;ll see the new virtual folder appear on the left Finder sidebar - <a href="#top">see the screenshot at the top of this post</a>.</p> <p>You can create additional Smart Folders for other useful file extensions, for example, VSCode/Positron project files (<em>.code-workspace</em> files).</p> <p>On Windows, I don&rsquo;t believe that there is an exact equivalent of a Smart Folder. I think that the closest you can get is to save a search in File Explorer. However, I find that in recent years File Explorer on Windows 11 has become incredibly slow and regularly crashes. Searching in <a href="https://files.community/" target="_blank" rel="noopener">Files</a> or <a href="https://filepilot.tech/" target="_blank" rel="noopener">File Pilot</a> is now much faster and more reliable.</p> <h2 id="summary">Summary</h2> <p>I have shown how to create a Smart Folder of your RStudio Project files for fast and convenient project launching on macOS.</p> Creating R, Python, Stata, and Julia tutorial worksheets (with and without solutions) using Quarto https://remlapmot.github.io/post/2025/quarto-conditional-content/ Tue, 18 Feb 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/quarto-conditional-content/ <h2 id="introduction">Introduction</h2> <p>I regularly need to produce exercises/tutorials for my students. One fantastic feature of R Markdown is that it allows me to create one R Markdown document which can be rendered to both the question document and the solutions document. This is achieved by toggling knitr chunk options such as eval, echo, and include, and using asis chunks to include the text for the solutions. I wrote a little package, <a href="https://remlapmot.github.io/knitexercise/" target="_blank" rel="noopener"><strong>knitexercise</strong></a> to help with this.</p> <p>The toggling of the knitr chunk options can be parameterised making it possible to have an R script which contains the code to conveniently produce both questions and solutions documents. An example R Markdown file, <em>exercise.Rmd</em>, might look like the following.</p> <pre><code class="language-plaintext">--- title: &quot;`r params$title`&quot; output: html_document params: solutions: TRUE title: &quot;Example exercise: Solutions&quot; --- ```{r setup, include=FALSE} knitr::opts_chunk$set(include = params$solutions) ``` 1. This is question 1. Which might have some R code you always want to show. ```{r, include=TRUE} # example code for the question ``` ```{asis} Paragraph text for the solution can be kept in the document in an `asis` chunk. And solution R code in an `r` chunk. Both of these will use the `include` value from the `setup` chunk. ``` ```{r} # example code for the solution ``` 2. This is question 2 ... </code></pre> <p>And the rendering script might look like this.</p> <pre><code class="language-r">rmarkdown::render(&quot;exercise.Rmd&quot;, output_file = &quot;exercise-questions&quot;, params = list(solutions = FALSE, title = &quot;Example exercise: Questions&quot;) ) rmarkdown::render(&quot;exercise.Rmd&quot;, output_file = &quot;exercise-solutions&quot; ) </code></pre> <h3 id="inline-code">Inline code</h3> <p>Inline code has been a <a href="https://bookdown.org/yihui/rmarkdown-cookbook/r-code.html" target="_blank" rel="noopener">feature of R Markdown</a> for a while. Yihui Xie and Christophe Dervieux have used it to pull off some fantastic tricks. My favourite trick is using it to programmatically write out R Markdown/Markdown code within an R Markdown document when a certain condition is met.</p> <pre><code class="language-plaintext">`r if (knitr::is_latex_output()) &quot;...some_Markdown_to_include_when_rendering_to_pdf...&quot;` </code></pre> <h3 id="the-problem">The problem</h3> <p>R Markdown is incredibly flexible because we can include R objects as document and chunk options. When translating this to a pure Quarto version we can do this in a document using in the knitr engine using the <a href="https://quarto.org/docs/tools/rstudio.html#knitr-engine" target="_blank" rel="noopener"><code>! expr ...</code> YAML tag literal</a>. As far as I am aware, this is not yet possible for any other Quarto engine. Despite this my aim was to see whether I could achieve parameterised conditional content inclusion/exclusion in a Quarto document using other engines.</p> <h2 id="programmatically-including-conditional-content-in-quarto-documents">Programmatically including conditional content in Quarto documents</h2> <p>The code described below is presented in the following <a href="https://github.com/remlapmot/quarto-parameterised-reports-test" target="_blank" rel="noopener">demo repo</a>, and the rendered output files can also <a href="https://remlapmot.github.io/quarto-parameterised-reports-test/" target="_blank" rel="noopener">be viewed</a>.</p> <h3 id="r-knitr-and-quarto">R: knitr and Quarto</h3> <p>I have already shown the R Markdown approach above.</p> <p>For Quarto using the knitr engine, in a blog post <a href="https://nrennie.rbind.io/blog/r-tutorial-worksheets-quarto/" target="_blank" rel="noopener">Nicola Rennie</a> used inline code to write out Quarto&rsquo;s <a href="https://quarto.org/docs/authoring/conditional.html" target="_blank" rel="noopener">conditional content classes</a>.</p> <pre><code class="language-plaintext">--- format: html params: hide_answers: true engine: knitr --- `r if (params$hide_answers) &quot;::: {.content-hidden}&quot;` Text and code for answers. `r if (params$hide_answers) &quot;:::&quot;` </code></pre> <p>Note that if you use the curly braces around the <code>r</code> to write you inline code then you need to enclose the output string in the <a href="https://quarto.org/docs/computations/inline-code.html#markdown-output" target="_blank" rel="noopener"><code>I()</code> function</a> and include the <code>else</code> statement as follows.</p> <pre><code class="language-plaintext">`{r} if (params$hide_answers) I(&quot;::: {.content-hidden}&quot;) else &quot;&quot;` </code></pre> <p>Then we can have a shell script to render our questions and solutions documents as follows.</p> <pre><code class="language-plaintext">quarto render exercise-r.qmd -o exercise-r-questions.html quarto render exercise-r.qmd -P hide_answers:false -o exercise-r-solutions.html </code></pre> <h3 id="python">Python</h3> <p>I realised I could adapt Nicola Rennie&rsquo;s approach for other engines. For the <code>jupyter: python3</code> engine we can do so as follows, the following code uses <a href="https://quarto.org/docs/computations/parameters.html#jupyter" target="_blank" rel="noopener">Quarto&rsquo;s parameters feature</a>.</p> <pre><code class="language-plaintext">--- format: html jupyter: python3 --- ```{python} #| include: false #| tags: [parameters] hide_answers = True ``` ```{python} #| include: false from IPython.display import Markdown ``` `{python} Markdown(&quot;::: {.content-hidden}&quot;) if hide_answers else Markdown(&quot; &quot;)` ```{python} print(&quot;Hidden in questions&quot;) ``` `{python} Markdown(&quot;:::&quot;) if hide_answers else Markdown(&quot; &quot;)` </code></pre> <p>The shell script to render our questions and solutions documents is as follows.</p> <pre><code class="language-plaintext">quarto render exercise-python.qmd -o exercise-python-questions.html quarto render exercise-python.qmd -P hide_answers:False -o exercise-python-solutions.html </code></pre> <p>As of Quarto version 1.7.23 it is possible to suppress the injected parameters cell.</p> <h3 id="stata">Stata</h3> <p>The Stata case involved two additional tricks. First, the <a href="https://hugetim.github.io/nbstata/" target="_blank" rel="noopener">nbstata Jupyter kernel</a> allows inline code, however this must be a <a href="https://www.stata.com/manuals/pdisplay.pdf" target="_blank" rel="noopener"><code>display</code> command</a>, so we cannot write the if statement within the inline code. I found I can overcome this by saving the different strings in scalars (because the inline code can&rsquo;t use local macros) at the top of the document as follows. Second, I didn&rsquo;t try but I suspect the nbstata kernel doesn&rsquo;t support parameters, and so I achieved the toggling of the code using an environment variable, e.g. <code>HIDE_ANSWERS_STATA</code>.</p> <pre><code class="language-plaintext">--- format: html jupyter: nbstata --- ```{stata} *| include: false local hide_answers : env HIDE_ANSWERS_STATA if (`hide_answers') { scalar hide_answers_open = &quot;::: {.content-hidden}&quot; scalar hide_answers_close = &quot;:::&quot; } else { scalar hide_answers_open = &quot; &quot; scalar hide_answers_close = &quot; &quot; } ``` `{stata} scalar(hide_answers_open)` ```{stata} display &quot;Hidden in questions&quot; ``` `{stata} scalar(hide_answers_close)` </code></pre> <p>The shell script to render the documents is then as follows, here we define the environment variable before the call to <code>quarto</code>.</p> <pre><code class="language-plaintext">HIDE_ANSWERS_STATA=1 quarto render exercise-stata.qmd -o exercise-stata-questions.html HIDE_ANSWERS_STATA=0 quarto render exercise-stata.qmd -o exercise-stata-solutions.html </code></pre> <h3 id="julia">Julia</h3> <p>For the native Julia engine I found that Quarto&rsquo;s parameterisation worked and that I didn&rsquo;t require a code chunk with a parameters tag.</p> <pre><code class="language-plaintext">--- format: html engine: julia --- ```{julia} #| include: false using Markdown ``` `{julia} hide_answers ? md&quot;::: {.content-hidden}&quot; : md&quot;&quot;` ```{julia} println(&quot;Hidden in questions&quot;) ``` `{julia} hide_answers ? md&quot;:::&quot; : md&quot;&quot;` </code></pre> <p>The shell script to render the documents is then as follows.</p> <pre><code class="language-plaintext">quarto render exercise-julia.qmd -P hide_answers:true -o exercise-julia-questions.html quarto render exercise-julia.qmd -P hide_answers:false -o exercise-julia-solutions.html </code></pre> <h2 id="problems-with-environment-variables">Problems with environment variables</h2> <p>I found that passing environment variables, to the <code>jupyter: python3</code> and <code>engine: julia</code> is unreliable/broken. Admittedly, I was not using Quarto in project mode with an <a href="https://quarto.org/docs/projects/environment.html" target="_blank" rel="noopener"><em>_environment</em> file</a>, but honestly it doesn&rsquo;t feel to me I should need to do that for a single document.</p> <p>The problem I found was that after a first render the value of the environment variable seems to be cached within the Quarto output document and I couldn&rsquo;t change it on subsequent renders. I also found that using the <a href="https://pypi.org/project/python-dotenv/" target="_blank" rel="noopener"><strong>dotenv</strong> package</a> to access was broken in the same way.</p> <p>For <code>engine: julia</code> I also found that passing environment variables is unreliable and like for the <code>jupyter: python3</code> engine I experienced environment variable values being stuck after the first render. However, using the Julia <a href="https://juliapackages.com/p/dotenv" target="_blank" rel="noopener"><strong>DotEnv</strong> package</a> did seem to be reliable.</p> <h2 id="summary">Summary</h2> <p>I have shown how to programmatically include conditional content for several Quarto engines (<code>knitr</code>, <code>jupyter: python3</code>, <code>jupyter: nbstata</code>, and <code>engine: julia</code>) using parameters or environment variables to toggle inline code to write Quarto Markdown in the Quarto documents. I use this to write exercise/tutorial documents in which a single Quarto document is used to output both the questions and solutions documents.</p> Creating effectively multi-engine Quarto documents using Quarto's embed shortcode https://remlapmot.github.io/post/2025/multi-engine-quarto/ Sat, 15 Feb 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/multi-engine-quarto/ <h2 id="introduction">Introduction</h2> <p>Have you ever needed to present the code and output for several languages in the same document or website? I work in (non-infectious disease) Epidemiology and so it is common that researchers would like to present R and Stata code in the same document. However, a Quarto document can only run a single engine. There are already several work around solutions, which include:</p> <ul> <li>writing out the different language code cells but making them unevaluated/not executed chunks (this is done alot on the <a href="https://quarto.org/docs/reference/" target="_blank" rel="noopener">Quarto documentation website</a>). One can also include saved plots from the different languages;</li> <li>if your document has a combination of languages from which you can call one from the other, such as using <a href="https://rstudio.github.io/reticulate/" target="_blank" rel="noopener"><strong>reticulate</strong></a> from within R to run Python, or using <a href="https://cran.r-project.org/package=Statamarkdown" target="_blank" rel="noopener"><strong>Statamarkdown</strong></a> from within R to run Stata, or using <a href="https://cran.r-project.org/package=JuliaCall" target="_blank" rel="noopener"><strong>JuliaCall</strong></a> from within R to run Julia, then these can be combined in a Quarto document;</li> <li>or for some languages like R and Python we could even embed full WebAssembly implementations of the language ( <a href="https://docs.r-wasm.org/webr/latest/" target="_blank" rel="noopener">WebR</a> and <a href="https://pyodide.org/en/stable/#" target="_blank" rel="noopener">Pyodide</a> respectively) within a webpage (which admittedly seems a little overkill for my work).</li> </ul> <p>I&rsquo;ve found an alternative solution allowing you to use the native engines for each language. I recently stumbled across <a href="https://quarto.org/docs/authoring/notebook-embed.html" target="_blank" rel="noopener">Quarto&rsquo;s embed shortcode</a>. This allows another (or selected cells from another) Quarto document to be embedded in a Quarto document. A thought occurred to me, what if the embedded Quarto document/s used a different engine? Would that work? This isn&rsquo;t explicitly mentioned on the documentation page, so I gave it a go. Remarkably, the answer turns out to be that it works! Let&rsquo;s find out what to do.</p> <h2 id="using-the-embed-shortcode-to-create-an-effectively-multi-engine-quarto-document">Using the embed shortcode to create an effectively multi-engine Quarto document</h2> <p>In the example below I&rsquo;m using a <a href="https://quarto.org/docs/output-formats/html-basics.html#tabsets" target="_blank" rel="noopener">tabset</a> in a html document using the knitr engine. We embed the documents using the alternative engines for Python, Stata, and Julia using the <code>{{&lt; embed &gt;}}</code> shortcode as shown below. For each language I just show printing a string and a basic plot.</p> <pre><code class="language-plaintext">--- title: An effectively multi-engine Quarto document using the embed shortcode format: html: embed-resources: true engine: knitr --- ::: {.panel-tabset .nav-pills} ## R ```{r} print(&quot;Hello World, from R&quot;) ``` ```{r} #| fig-align: &quot;center&quot; x &lt;- seq(-10,10, by = 0.1) y &lt;- x ^ 3 plot(x, y, type = &quot;l&quot;) ``` ## Python {{&lt; embed python-code-using-jupyter-python3-engine.qmd echo=true &gt;}} ## Stata {{&lt; embed stata-code-using-jupyter-nbstata-engine.qmd echo=true &gt;}} ## Julia {{&lt; embed julia-code-using-julia-engine.qmd echo=true &gt;}} ::: </code></pre> <ul> <li>the <em>python-code-using-jupyter-python3-engine.qmd</em> document uses the <code>jupyter: python3</code> engine ( <a href="https://quarto.org/docs/computations/python.html" target="_blank" rel="noopener">documentation for using Python in Quarto</a>);</li> <li>the <em>stata-code-using-jupyter-nbstata-engine.qmd</em> document uses the <code>jupyter: nbstata</code> engine ( <a href="https://hugetim.github.io/nbstata/" target="_blank" rel="noopener">documentation for nbstata</a>);</li> <li>and the <em>julia-code-using-julia-engine.qmd</em> uses <code>engine: julia</code>. Alternatively, it should be possible to use the <a href="https://julialang.github.io/IJulia.jl/stable/" target="_blank" rel="noopener">IJulia</a> Jupyter kernel ( <a href="https://quarto.org/docs/computations/julia.html" target="_blank" rel="noopener">documentation for using Julia in Quarto</a>)</li> </ul> <p>Of course, I assume that you have setup each engine beforehand.</p> <p>Rendering the Quarto document above results in the embedded documents being executed and embedded within it. I&rsquo;ve included the output below (and the full source code is in <a href="https://github.com/remlapmot/quarto-multi-engine-using-embed-example" target="_blank" rel="noopener">this repository</a>). Click the tabs to show the code and output for each language.</p> <p align="center"> <iframe src="https://remlapmot.github.io/quarto-multi-engine-using-embed-example/" height="1075" width="800"> </iframe> </p> <p>In the code above, in each case, I embed the whole Quarto document but you can also specify a specific code block id (or if the embedded document is a Jupyter Notebook, <em>.ipynb</em> file, you can specify a cell id, label, or tag).</p> <h2 id="summary">Summary</h2> <p>I have shown how to use the Quarto embed shortcode to embed Quarto documents using alternative engines to create an effectively multi-engine Quarto document.</p> Seven tips for creating Quarto revealjs presentations https://remlapmot.github.io/post/2025/quarto-revealjs-tips/ Sun, 19 Jan 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/quarto-revealjs-tips/ <h2 id="introduction">Introduction</h2> <p>Like a lot of lecturers I&rsquo;m teaching at the moment. If I have a presentation to do that&rsquo;s mainly images I&rsquo;ll use Powerpoint or Google Slides. If the presentation includes maths or code or both I used to use LaTeX Beamer. Over the years I have grown tired of Beamer and so I thought I&rsquo;d try making some revealjs presentations using Quarto.</p> <p>Revealjs has been around far longer than Quarto, I remember seeing a colleague present using revealjs in about 2012, and you can happily make a revealjs presentation without Quarto. However, the convenience of writing in the Quarto Markdown format is fantastic.</p> <p>To learn the basics of making revealjs slides with Quarto I recommend reading <a href="https://quarto.org/docs/presentations/revealjs/" target="_blank" rel="noopener">the user guide</a>. And full documentation is available <a href="https://quarto.org/docs/reference/formats/presentations/revealjs.html" target="_blank" rel="noopener">here</a> and <a href="https://quarto.org/docs/presentations/revealjs/advanced.html" target="_blank" rel="noopener">here</a>.</p> <p>The following tips are solutions to little hurdles that I found I needed to overcome when making my first revealjs presentations using Quarto.</p> <h2 id="1-slide-size-and-testing-on-a-smaller-display-size-on-your-own-monitor">1. Slide size and testing on a smaller display size on your own monitor</h2> <p>The great thing about making a presentation in Powerpoint or Google Slides is that you can instantly see if your content fits on a slide. So it helps to have an understanding of how big a slide is. By default a revealjs slide is <a href="https://quarto.org/docs/presentations/revealjs/advanced.html#presentation-size" target="_blank" rel="noopener">1050px wide by 700px tall</a>. This helps you when setting the size of figures and code blocks.</p> <p>Additionally it is helpful to be able to preview your slides on the size on monitor you will be presenting on. My University has 1080p monitors in most lecture theatres, however, my monitor is bigger than that. Chrome allows setting the size of the display by opening Developer Tools (Three dots | More tools | Developer Tools). Then you can enter the desired resolution in the top bar as follows (a 1080p screen is 1080 pixels tall and 1920 pixels wide).</p> <img src="https://remlapmot.github.io/post/2025/quarto-revealjs-tips/img/slide-size-and-1080p.png" alt="Screenshot of using Google Chrome developer tools to inspect the size of a slide and to preview the slides at 1080p resolution." width="700" style="display: block; margin: auto;"> <p>I find this especially helpful when working on a Mac, whose laptops often have unusual screen resolutions.</p> <h2 id="2-wider-width-cutoff-in-code-chunks">2. Wider width cutoff in code chunks</h2> <p>Vertical space on a slide is at a premium. Therefore, in a presentation I line break my code wider that I do when coding normally. As described in the <a href="https://bookdown.org/yihui/rmarkdown-cookbook/opts-tidy.html" target="_blank" rel="noopener">RMarkdown cookbook</a> and in <a href="https://github.com/quarto-dev/quarto-cli/issues/5852" target="_blank" rel="noopener">this GitHub issue</a> we can enable the use of the <strong>formatR</strong> package on a code chunk using the <code>tidy</code> and <code>tidy-opts</code> chunk options (I think the default is 85 characters, so I choose a value greater than that. I find that 110 is about the widest I can set).</p> <pre><code class="language-markdown">```{r} #| tidy: TRUE #| tidy-opts: !expr list(width.cutoff = I(110)) # ... Wide R code here ... ``` </code></pre> <p>We can also set these globally for the Quarto document in either the YAML header,</p> <pre><code class="language-yaml">knitr: opts_chunk: tidy: TRUE tidy-opts: !expr list(width.cutoff = I(110)) </code></pre> <p>or in a chunk at the top of your Quarto document.</p> <pre><code class="language-markdown">```{r} #| include: false knitr::opts_chunk$set(tidy = TRUE, tidy.opts = list(width.cutoff = I(110))) ``` </code></pre> <h2 id="3-taller-code-chunks">3. Taller code chunks</h2> <p>Code chunks come in several flavours in a rendered presentation. There are code input chunks, code output chunks, and chunks for any code output errors. To make the input code chunks taller there is a convenient <a href="https://quarto.org/docs/presentations/revealjs/#code-block-height" target="_blank" rel="noopener">Quarto option <code>code-block-height</code></a> which is specified in the YAML header (the default is 500px, so pick a value greater than that).</p> <pre><code class="language-yaml">--- format: revealjs: code-block-height: 650px --- </code></pre> <h2 id="4-taller-code-output-chunks">4. Taller code output chunks</h2> <p>Currently, there doesn&rsquo;t seem to be a Quarto option to make the code output chunks taller. Therefore I had to inspect the source code of the html slides to find out how to modify this. In most browsers simply right click on a slide and in Chrome click <em>Inspect</em> - do this over the element on the page you want to find out about.</p> <img src="https://remlapmot.github.io/post/2025/quarto-revealjs-tips/img/chrome-context-menu-inspect.png" alt="Screenshot of Google Chrome context menu." width="600" style="display: block; margin: auto;"> <p>Doing so over a code output cell shows the developer tools on the right handside. We can then see that the elements and classes for the code output cell are given just above the bottom right pane, i.e., <code>div.cell div.cell-output.cell-output-stdout pre code</code>.</p> <img src="https://remlapmot.github.io/post/2025/quarto-revealjs-tips/img/inspect-output-cell.png" alt="Screenshot of using Google Chrome developer tools to inspect the CSS class of an element of a code output cell." width="500" style="display: block; margin: auto;"> <p>Therefore, we can make the code output cell have a taller maximum height by specifying <code>max-height</code> as follows in a CSS file which we reference in the YAML header. Note that the default is 400px, so choose a value larger than that.</p> <pre><code class="language-yaml">--- format: revealjs: css: custom.css --- </code></pre> <p>The contents of <em>custom.css</em> use the elements and classes of the code chunk we discovered as follows.</p> <pre><code class="language-css">.cell .cell-output-stdout pre code { max-height: 650px; // Adjust this value as needed overflow-y: auto; // Add this to handle overflow } </code></pre> <h2 id="5-embedding-interactive-mentimeter-presentations">5. Embedding interactive Mentimeter presentations</h2> <p>Mentimeter provides the html code required to embed a presentation within an html document. To obtain this, go into a presentation and click the <em>Share</em> button, then select the <em>Slides</em> tab, then click <em>Copy code</em> under <em>Embed slides</em>.</p> <img src="https://remlapmot.github.io/post/2025/quarto-revealjs-tips/img/mentimeter-share-iframe-code.png" alt="Screenshot of the menu to copy the html code to embed a Mentimeter presentation." width="700" style="display: block; margin: auto;"> <p>This results in a div containing an iframe, which we can simply paste in as the content of our slide (I have added the line breaks and replaced part of the URL to my presentation with hashes).</p> <pre><code class="language-markdown">## My slide with an embedded Menti &lt;div style='position: relative; padding-bottom: 56.25%; padding-top: 35px; height: 0; overflow: hidden;'&gt; &lt;iframe sandbox='allow-scripts allow-same-origin allow-presentation' allowfullscreen='true' allowtransparency='true' frameborder='0' height='315' src='https://www.mentimeter.com/app/presentation/###############/embed' style='position: absolute; top: 0; left: 0; width: 100%; height: 100%;' width='420'&gt; &lt;/iframe&gt; &lt;/div&gt; </code></pre> <p>But when I first rendered a presentation using this code I didn&rsquo;t see my Mentimeter presentation in the relevant slide but rather the following spinning dots.</p> <img src="https://remlapmot.github.io/post/2025/quarto-revealjs-tips/img/menti-failing-to-load.gif" alt="Animated GIF of the spinning dots when a Mentimeter presentation is incorrectly embedded in an HTML document." width="170" style="display: block; margin: auto;"> <p>Thanks <a href="https://stackoverflow.com/questions/79092017/quarto-markdown-to-revealjs-or-html-desmos-embedded-iframes-not-loading" target="_blank" rel="noopener">to this post</a> it turns out this is because I had specified <code>embed-resources: true</code> in my revealjs options in the YAML header. When you do this you need to add the <code>data-external=&quot;1&quot;</code> attribute to the iframe <a href="https://quarto.org/docs/reference/formats/presentations/revealjs.html#rendering" target="_blank" rel="noopener">as detailed in the Quarto documentation</a> as follows.</p> <pre><code class="language-html">&lt;iframe data-external=&quot;1&quot; sandbox=... </code></pre> <p>And then my Mentimeter presentation was correctly embedded in the slide as per the screenshot at the <a href="#top">top of this post</a>.</p> <h2 id="6-disable-html-table-processing-for-some-tables">6. Disable HTML table processing for some tables</h2> <p>There are now so many fantastic R packages for html table generation. My two favourite are <a href="https://www.danieldsjoberg.com/gtsummary/" target="_blank" rel="noopener"><strong>gtsummary</strong></a> and <a href="https://strengejacke.github.io/sjPlot/" target="_blank" rel="noopener"><strong>sjPlot</strong></a>. However, I notice that for some <strong>sjPlot</strong> tables Quarto issues the following warning when rendering the slides.</p> <pre><code class="language-plaintext">WARNING (/Applications/quarto/share/filters/main.lua:9319) Unable to parse table from raw html block: skipping. </code></pre> <p>To avoid this it&rsquo;s possible to <a href="https://quarto.org/docs/authoring/tables.html#disabling-quarto-table-processing" target="_blank" rel="noopener">disable Quarto&rsquo;s html table processing</a> using the <code>html-table-processing</code> argument either at the document or chunk level. Here is an example slide doing so at the chunk level.</p> <pre><code class="language-markdown">## Slide presenting a multilevel model ```{r} #| html-table-processing: none library(sjPlot) library(lme4) fm1 &lt;- lmer(Reaction ~ Days + (Days | Subject), sleepstudy) tab_model(fm1) ``` </code></pre> <h2 id="7-programmatic-rendering-and-pdf-export-for-printing">7. Programmatic rendering and pdf export for printing</h2> <p>To render a Quarto document we can click the <em>Render</em> button in RStudio, but I find it easier to make a <em>render.R</em> script with a call to the <a href="https://quarto-dev.github.io/quarto-r/" target="_blank" rel="noopener"><strong>quarto</strong></a> package&rsquo;s <code>quarto_render()</code> function.</p> <p>It&rsquo;s helpful to provide students with a pdf copy of the slides for printing. You can do this in a browser through the print menu or programmatically using <strong>pagedown</strong>&rsquo;s <code>chrome_print()</code> function. When I called this function with no options the page size of the pdf was unusual, producing pages with neither A4 nor US paper size, so I call it as follows to ensure an A4 page size.</p> <p>Therefore, my <em>render.R</em> script often looks something like the following.</p> <pre><code class="language-r">quarto::quarto_render(input = &quot;my-great-slides.qmd&quot;) pagedown::chrome_print(&quot;my-great-slides.html&quot;, options = list( printBackground = FALSE, preferCSSPageSize = FALSE, paperWidth = 8.3, paperHeight = 11.7, marginTop = 0.1, marginBottom = 0.1, marginLeft = 0.1, marginRight = 0.1)) </code></pre> <p>I usually find that a few slides have slightly too much content for an A4 page, and so those slides will take up 2 pages in the pdf. And hence the pdf usually has a few more pages than the number of slides.</p> <h2 id="summary">Summary</h2> <p>I have shown seven tips that I needed to workout when making revealjs presentations with Quarto.</p> My #GitHubUnwrapped 2024! https://remlapmot.github.io/post/2025/github-unwrapped-2024/ Wed, 08 Jan 2025 00:00:00 +0000 https://remlapmot.github.io/post/2025/github-unwrapped-2024/ <p>My #GitHubUnwrapped 2024!</p> <p align="center"><iframe src="https://drive.google.com/file/d/10zFAtzcEFhxLNSRDZDXUgnafU4GDVepZ/preview" width="640" height="480" allow="autoplay"></iframe></p> <p>Made with <a href="https://www.githubunwrapped.com/" target="_blank" rel="noopener">https://www.githubunwrapped.com/</a>.</p> Launch RStudio, Positron, and other Data Science apps from your Finder Toolbar https://remlapmot.github.io/post/2024/finder-toolbar-apps/ Mon, 23 Dec 2024 00:00:00 +0000 https://remlapmot.github.io/post/2024/finder-toolbar-apps/ <h2 id="introduction">Introduction</h2> <p>Have you ever wanted to be able to quickly open a Data Science app, say RStudio Desktop or Positron in the current Finder window at the click of a button? We&rsquo;ll see how to achieve this by creating Automator apps. Here&rsquo;s a screenshot of what we&rsquo;ll end up with.</p> <img src="https://remlapmot.github.io/post/2024/finder-toolbar-apps/img/finder-window-with-toolbar-apps.png" alt="Screenshot of a Finder window with app icons in its toolbar." width="630" style="display: block; margin: auto;"> <h2 id="creating-the-automator-apps">Creating the Automator apps</h2> <p>Open Automator and select <em>New</em> then <em>Application</em> and click <em>Choose</em>. Then in the top left search bar enter <em>applescript</em> and drag and drop the <em>Run Applescript</em> action onto the main window.</p> <img src="https://remlapmot.github.io/post/2024/finder-toolbar-apps/img/automator-run-applescript.png" alt="Screenshot of the Automator app." width="630" style="display: block; margin: auto;"> <p>We then need to enter the relevant AppleScript code for launching each app in the current Finder window. Currently, I use <a href="https://wezfurlong.org/wezterm/" target="_blank" rel="noopener">WezTerm</a> for my terminal emulator, <a href="https://zed.dev/" target="_blank" rel="noopener">Zed</a> as my main text editor, <a href="https://posit.co/products/open-source/rstudio/" target="_blank" rel="noopener">RStudio Desktop</a> for most of my R/R Markdown/Quarto coding, <a href="https://code.visualstudio.com/" target="_blank" rel="noopener">Visual Studio Code</a> for other text editing tasks, and I have been starting to try out <a href="https://positron.posit.co/" target="_blank" rel="noopener">Positron</a>. My AppleScript code for each app is as follows.</p> <h3 id="rstudio-desktop">RStudio Desktop</h3> <p>It is worth noting that RStudio automatically detects whether there is an <em>.Rproj</em> file in the directory and opens in project mode if one is found (note this only works if RStudio is not already open).</p> <pre><code class="language-applescript">on run {input, parameters} tell application &quot;Finder&quot; set myWin to window 1 set thePath to (quoted form of POSIX path of (target of myWin as alias)) do shell script &quot;/Applications/RStudio.app/Contents/MacOS/RStudio &quot; &amp; thePath end tell end run </code></pre> <p>We can create an alternative app which specifically opens RStudio project (<em>.Rproj</em>) files as follows.</p> <pre><code class="language-applescript">on run {input, parameters} tell application &quot;Finder&quot; try set currentFolder to (folder of window 1) as alias set workspaceFiles to (every file of currentFolder whose name extension is &quot;Rproj&quot;) if (count of workspaceFiles) = 0 then set thePath to quoted form of POSIX path of (currentFolder as alias) do shell script &quot;/Applications/RStudio.app/Contents/MacOS/RStudio &quot; &amp; thePath else if (count of workspaceFiles) = 1 then set workspaceFile to item 1 of workspaceFiles set workspacePath to POSIX path of (workspaceFile as alias) do shell script &quot;open -n -a RStudio &quot; &amp; quoted form of workspacePath else if (count of workspaceFiles) &gt; 1 then display dialog &quot;Multiple Rproj files found in directory.&quot; end if on error display dialog &quot;No Finder window is open.&quot; end try end tell end run </code></pre> <h3 id="wezterm">WezTerm</h3> <pre><code class="language-applescript">on run {input, parameters} tell application &quot;Finder&quot; set myWin to window 1 set thePath to (quoted form of POSIX path of (target of myWin as alias)) do shell script &quot;/opt/homebrew/bin/wezterm-gui start --cwd &quot; &amp; thePath &amp; &quot;&amp;&gt; /dev/null &amp;&quot; end tell end run </code></pre> <h3 id="zed">Zed</h3> <pre><code class="language-applescript">on run {input, parameters} tell application &quot;Finder&quot; set myWin to window 1 set thePath to (quoted form of POSIX path of (target of myWin as alias)) do shell script &quot;/usr/local/bin/zed -n &quot; &amp; thePath end tell end run </code></pre> <h3 id="r-launched-in-a-wezterm-terminal-session">R launched in a WezTerm terminal session</h3> <pre><code class="language-applescript">on run {input, parameters} tell application &quot;Finder&quot; set myWin to window 1 set thePath to (quoted form of POSIX path of (target of myWin as alias)) do shell script &quot;/opt/homebrew/bin/wezterm-gui start --cwd &quot; &amp; thePath &amp; &quot; -- /usr/local/bin/R &amp;&gt; /dev/null &amp;&quot; end tell end run </code></pre> <h3 id="visual-studio-code-and-positron">Visual Studio Code and Positron</h3> <p>First enable the ability to launch these apps with <code>code</code> and <code>positron</code> from a Terminal in each app, see <a href="https://code.visualstudio.com/docs/setup/mac#_launching-from-the-command-line" target="_blank" rel="noopener">here</a> and <a href="https://positron.posit.co/add-to-path.html" target="_blank" rel="noopener">here</a>.</p> <p>This script is more involved because we first check for any <em>.code-workspace</em> files and open one if found. My AppleScript coding is not very proficient, so there may more efficient approaches to coding this. If we didn&rsquo;t explicitly open the <em>.code-workspace</em> file and if one is present in the directory Visual Studio Code/Positron will detect it and pop up a dialogue box asking if we want to reopen the directory as a workspace (I just prefer to skip this step).</p> <pre><code class="language-applescript">on run {input, parameters} tell application &quot;Finder&quot; try set currentFolder to (folder of window 1) as alias set workspaceFiles to (every file of currentFolder whose name extension is &quot;code-workspace&quot;) if (count of workspaceFiles) = 0 then set folderPath to POSIX path of currentFolder do shell script &quot;/usr/local/bin/positron -n &quot; &amp; quoted form of folderPath else if (count of workspaceFiles) = 1 then set workspaceFile to item 1 of workspaceFiles set workspacePath to POSIX path of (workspaceFile as alias) do shell script &quot;/usr/local/bin/positron -n &quot; &amp; quoted form of workspacePath else if (count of workspaceFiles) &gt; 1 then display dialog &quot;Multiple code-workspace files found in directory.&quot; end if on error display dialog &quot;No Finder window is open.&quot; end try end tell end run </code></pre> <p>(For your Visual Studio Code app simply replace <code>positron</code> with <code>code</code>.)</p> <h3 id="ghostty">Ghostty</h3> <pre><code class="language-applescript">on run {input, parameters} tell application &quot;Finder&quot; set myWin to window 1 set thePath to (quoted form of POSIX path of (target of myWin as alias)) do shell script &quot;open -na ghostty --args --title=ghostty-from-finder --working-directory=&quot; &amp; thePath end tell end run </code></pre> <h3 id="saving-and-adding-icons">Saving and adding icons</h3> <p>After adding the AppleScript code save each app. I call mine, e.g., <em>RStudio-openhere.app</em>.</p> <p>Next it is helpful to give each app the relevant icon. To do this in Finder bring up the Info boxes for the original app and your <em>-openhere</em> version by selecting each app and pressing <kbd>Cmd</kbd>+<kbd>I</kbd>. Next drag the large icon from the original app onto the small icon of your <em>-openhere</em> app.</p> <img src="https://remlapmot.github.io/post/2024/finder-toolbar-apps/img/updating-icon.png" alt="Screenshot of copying the Positron icon onto our openhere app." width="630" style="display: block; margin: auto;"> <h3 id="adding-the-apps-to-the-finder-toolbar">Adding the apps to the Finder toolbar</h3> <p>Finally, we need to place shortcuts of these apps onto the Finder toolbar. To do this first hold down <kbd>Cmd</kbd> then drag the app from Finder onto the toolbar to the location you would like. On release the app should now be in the toolbar. (And to remove an icon from the toolbar, again hold <kbd>Cmd</kbd> then drag it off the toolbar.)</p> <img src="https://remlapmot.github.io/post/2024/finder-toolbar-apps/img/add-app-to-toolbar.png" alt="Screenshot of moving our Positron-openhere app onto the Finder toolbar." width="630" style="display: block; margin: auto;"> <h2 id="summary">Summary</h2> <p>I have shown how to create Automator apps to open RStudio Desktop, Positron, and several other Data Science apps from the current Finder window.</p> The MRC IEU R-Universe of Mendelian randomization related R packages ๐Ÿš€ https://remlapmot.github.io/talk/2024_mr/ Wed, 19 Jun 2024 16:45:00 +0000 https://remlapmot.github.io/talk/2024_mr/ Supercharge your #rstats web searching in Google Chrome with Site Search Shortcuts https://remlapmot.github.io/post/2024/chrome-site-search-shortcuts/ Fri, 14 Jun 2024 00:00:00 +0000 https://remlapmot.github.io/post/2024/chrome-site-search-shortcuts/ <h2 id="introduction">Introduction</h2> <p>Have you ever typed <code>Amazon</code> into the Google Chrome address bar and seen the address bar indicate that it&rsquo;s now searching the Amazon site? It turns out this is a feature in Google Chrome called site search shortcuts.</p> <p>We can see what default shortcuts Chrome provides by in the Chrome address bar going to <code>chrome://settings/searchEngines</code> and scrolling to the <em>Site Search</em> section.</p> <p>From here we can see that we can define our own shortcuts, so let&rsquo;s define some helpful ones related to R and statistics.<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup></p> <h2 id="helpful-google-chrome-site-search-shortcuts-for-r-and-statistics">Helpful Google Chrome site search shortcuts for R and statistics</h2> <p> <a href="https://r-pkg.org/" target="_blank" rel="noopener">METACRAN</a> provides several amazingly useful services around CRAN.</p> <ul> <li>A shortcut for searching the METACRAN CRAN mirror on GitHub</li> </ul> <pre><code class="language-plaintext">Name: CRAN mirror on GitHub Shortcut: @cran URL with %s in place of query: https://github.com/search?q=user%3Acran%20%s&amp;ref=opensearch&amp;type=code </code></pre> <p>On the Chrome settings page click add and enter the information. To use this simply type <code>@cran</code> into the address bar</p> <img src="https://remlapmot.github.io/post/2024/chrome-site-search-shortcuts/img/cran-shortcut-step1.png" alt="Screenshot of a site search shortcut in the Google Chrome address bar." width="630" style="display: block; margin: auto;"> <p>and then type your search term</p> <img src="https://remlapmot.github.io/post/2024/chrome-site-search-shortcuts/img/cran-shortcut-step2.png" alt="Screenshot of a site search shortcut in the Google Chrome address bar." width="630" style="display: block; margin: auto;"> <p>Here are some other shortcuts.</p> <ul> <li>A shortcut for searching for a package description on METACRAN</li> </ul> <pre><code class="language-plaintext">Name: METACRAN Shortcut: @metacran URL with %s in place of query: https://r-pkg.org/search.html/?q=%s </code></pre> <ul> <li>A shortcut for searching using <a href="https://rseek.org/" target="_blank" rel="noopener">Rseek</a></li> </ul> <pre><code class="language-plaintext">Name: Rseek Shortcut: @rseek URL with %s in place of query: https://rseek.org/?q=%s </code></pre> <ul> <li>A shortcut for searching <a href="https://r-universe.dev/" target="_blank" rel="noopener">R-universe</a></li> </ul> <pre><code class="language-plaintext">Name: R-universe Shortcut: @runi URL with %s in place of query: https://r-universe.dev/search/?q=%s </code></pre> <ul> <li>A shortcut for searching the third edition of the Oxford Dictionary of Statistics</li> </ul> <pre><code class="language-plaintext">Name: Oxford Dictionary of Statistics Shortcut: @stats URL with %s in place of query: https://www.oxfordreference.com/search?source=%2F10.1093%2Facref%2F9780199679188.001.0001%2Facref-9780199679188&amp;q=%s </code></pre> <p>And we could define many more.</p> <h2 id="summary">Summary</h2> <p>In summary we have defined several Google Chrome site search shortcuts related to R and statistics.</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>I am following an excellent <a href="https://chromeunboxed.com/chrome-site-search-shortcuts" target="_blank" rel="noopener">blogpost</a> by Chrome Unboxed.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div> Running MLwiN using mlnscript via the R2MLwiN R package on Apple Silicon Macs https://remlapmot.github.io/post/2024/mlnscript-r2mlwin-apple-silicon/ Tue, 02 Apr 2024 00:00:00 +0000 https://remlapmot.github.io/post/2024/mlnscript-r2mlwin-apple-silicon/ <h2 id="introduction">Introduction</h2> <p>MLwiN from the Centre for Multilevel Modelling (CMM) at the University of Bristol (disclaimer: where I also work) is a fantastic piece of software ( <a href="#ref-mlwin">Charlton et al. 2024</a>). The name suggests it only works on Windows, but as weโ€™ll find out this is very much not the case.</p> <p>However, in the past this was sort of true because to make it work on a Mac (or Linux machine) with an Intel processor one would need to run it using <a href="https://www.winehq.org/" target="_blank" rel="noopener">Wine</a>.</p> <p>More recently, CMM have cleverly made the MLwiN libraries available for other operating systems in a command line version of the program called <code>mlnscript</code> and an accompanying library. The files for macOS are universal binaries which means that they run natively on both Intel and Apple Silicon Macs. Letโ€™s find out how to set this up.<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup></p> <h2 id="setting-up-mlnscript-on-an-apple-silicon-mac">Setting up mlnscript on an Apple Silicon Mac</h2> <ul> <li> <p>Obtain the installer for macOS. See the relevant download page (depending upon whether you are an academic) on the MLwiN <a href="https://www.bristol.ac.uk/cmm/software/mlwin/" target="_blank" rel="noopener">website</a>. On the form on the <em>File to download</em> dropdown menu select the <em>mlnscript for MacOS</em> option. This will give you the <em>MLN.dmg</em> installer.</p> <img src="https://remlapmot.github.io/post/2024/mlnscript-r2mlwin-apple-silicon/img/mlwin-download-choice.png" alt="Screenshot of MLwiN download choices." width="350" style="display: block; margin: auto;"> </li> <li> <p>Double click the installer. On macOS it is recommended to install the files into the <em>/opt/mln/</em> directory, which you will need to create with Admin permissions, or install to another directory if you donโ€™t have Admin permissions. Copy the 2 files <em>mlnscript</em> and <em>libmln.dylib</em> into the <em>/opt/mln</em> (or other) directory.</p> </li> <li> <p>Once installed we can check that <em>mlnscript</em> and <em>libmln.dylib</em> are universal binaries as follows (we could also use the <code>file</code> command).</p> <pre><code class="language-bash">lipo -archs /opt/mln/mlnscript ## x86_64 arm64 </code></pre> <p>Since both architectures are listed in the output this indicates the files are universal binaries. Apple Silicon Macs will use the arm64 architecture.</p> </li> <li> <p>Now we need to grant the two files permission to run. To do this run the following in your Terminal.</p> <pre><code class="language-bash">/opt/mln/mlnscript --version </code></pre> <p>On first run, this will fail with a pop-up similar to the following.</p> <img src="https://remlapmot.github.io/post/2024/mlnscript-r2mlwin-apple-silicon/img/security-popup-01.png" alt="Screenshot of macOS security warning pop-up." width="350" style="display: block; margin: auto;"> <p>Click <em>Cancel</em> and then go into the <em>System settings</em> | <em>Privacy &amp; Security</em> and scroll down and click <em>Allow Anyway</em>.</p> <img src="https://remlapmot.github.io/post/2024/mlnscript-r2mlwin-apple-silicon/img/mlnscript-settings.png" alt="Screenshot of macOS privacy and security setting." style="display: block; margin: auto;"> <p>Then running the version check command again you may receive another popup in which you click <em>Open</em>.</p> <img src="https://remlapmot.github.io/post/2024/mlnscript-r2mlwin-apple-silicon/img/security-popup-02.png" alt="Screenshot of macOS security warning pop-up." width="350" style="display: block; margin: auto;"> </li> </ul> <p>After this the first popup will then appear but about the <em>libmln.dylib</em> file. Again set the security setting to <em>Allow All</em>.</p> <p>Now running the version check command again you should see the version number โ€“ which is currently 3.10.</p> <pre><code class="language-bash">/opt/mln/mlnscript --version ## 3.13 </code></pre> <ul> <li> <p>In R we then install the <strong>R2MLwiN</strong> package from CRAN ( <a href="#ref-r2mlwin">Zhang et al. 2016</a>).</p> <pre><code class="language-r">install.packages(&quot;R2MLwiN&quot;) </code></pre> </li> </ul> <p>This completes the setup - phew ๐Ÿ˜ฎ!</p> <h2 id="running-a-multilevel-model">Running a multilevel model</h2> <p>For an example we could run one of the demos in the package, we can list those with the following code.</p> <pre><code class="language-r">demo(package = &quot;R2MLwiN&quot;) </code></pre> <p>We can run one, for example, letโ€™s fit the random intercept model from the <em>UserGuide02</em> demo.</p> <pre><code class="language-r">library(R2MLwiN) # if you did not install mlnscript and libmln.dylib in /opt/mln , set: # options(MLwiN_path = &quot;/path-to/mlnscript&quot;) (mymodel1 &lt;- runMLwiN(normexam ~ 1 + sex + (1 | student), data = tutorial)) #&gt; #&gt; -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- #&gt; MLwiN (version: unknown or &gt;3.09) multilevel model (Normal) #&gt; Estimation algorithm: IGLS Elapsed time : 0.03s #&gt; Number of obs: 4059 (from total 4059) The model converged after 3 iterations. #&gt; Log likelihood: -5727.9 #&gt; Deviance statistic: 11455.7 #&gt; --------------------------------------------------------------------------------------------------- #&gt; The model formula: #&gt; normexam ~ 1 + sex + (1 | student) #&gt; Level 1: student #&gt; --------------------------------------------------------------------------------------------------- #&gt; The fixed part estimates: #&gt; Coef. Std. Err. z Pr(&gt;|z|) [95% Conf. Interval] #&gt; Intercept -0.14035 0.02463 -5.70 1.209e-08 *** -0.18862 -0.09208 #&gt; sexgirl 0.23367 0.03179 7.35 1.985e-13 *** 0.17136 0.29598 #&gt; Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 #&gt; --------------------------------------------------------------------------------------------------- #&gt; The random part estimates at the student level: #&gt; Coef. Std. Err. #&gt; var_Intercept 0.98454 0.02185 #&gt; -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*- </code></pre> <p>We can see the output is in several sections. The first section tells us about how <code>mlnscript</code>, which estimation algorithm it used, how long it took to fit the model, and some characteristics of the dataset. The second section tells us about the model, in this case a random intercept model. The third section is the fixed effect estimates and the associated statistical inference for them. The fourth section is the random effect variance estimates.</p> <p>And we can continue with our multilevel modelling as we like.</p> <h2 id="summary">Summary</h2> <p>Despite having <em>Win</em> in its name, MLwiN is available as a command line program, <code>mlnscript</code>, which is available on operating systems other than Windows (and indeed with other architectures), including macOS for both Intel and Apple Silicon processors and various Linux and Unix distributions (CentOS, Debian, Fedora, FreeBSD, Rocky, and Ubuntu). This is straightforward to use from R via the <strong>R2MLwiN</strong> package.</p> <h2 id="references">References</h2> <div id="refs" class="references csl-bib-body hanging-indent" entry-spacing="0"> <div id="ref-mlwin" class="csl-entry"> <p>Charlton, C., J. Rasbash, W. J. Browne, M. Healy, and B. Cameron. 2024. <em>MLwiN Version 3.10</em>. Bristol, UK: Centre for Multilevel Modelling, University of Bristol. <a href="https://www.bristol.ac.uk/cmm/software/mlwin/" target="_blank" rel="noopener">https://www.bristol.ac.uk/cmm/software/mlwin/</a>.</p> </div> <div id="ref-r2mlwin" class="csl-entry"> <p>Zhang, Z., R. M. A. Parker, C. M. J. Charlton, G. Leckie, and W. J. Browne. 2016. โ€œR2MLwiN: A Package to Run MLwiN from Within R.โ€ <em>Journal of Statistical Software</em> 72 (10): 1โ€“43. <a href="https://doi.org/10.18637/jss.v072.i10" target="_blank" rel="noopener">https://doi.org/10.18637/jss.v072.i10</a>.</p> </div> </div> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>This post is essentially a more detailed explanation of the advice given on the MLwiN website, <a href="https://www.bristol.ac.uk/cmm/software/support/support-faqs/commands-macros.html" target="_blank" rel="noopener">here</a> and <a href="https://www.bristol.ac.uk/cmm/software/mlwin/features/sysreq.html#unix" target="_blank" rel="noopener">here</a>.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div> Creating a simple Automator app to launch a new instance of RStudio Desktop on macOS https://remlapmot.github.io/post/2024/macos-rstudio-another/ Sun, 31 Mar 2024 00:00:00 +0000 https://remlapmot.github.io/post/2024/macos-rstudio-another/ <h2 id="introduction-how-to-open-a-second-instance-of-rstudio-desktop-on-macos">Introduction: how to open a second instance of RStudio Desktop on macOS</h2> <p>On macOS when RStudio Desktop is open it can be inconvenient to open a second (or third or fourth) instance of it.</p> <p>If you left click the RStudio Desktop icon nothing happens. If you right click on the RStudio Desktop icon there is a convenient <em>New RStudio Window</em>. The problem with this is that by default if you have an RStudio project open the new RStudio window is also opened in that project. However, that can be overcome by opening the RStudio preference pane and in the first tab (<em>General</em> | <em>Basic</em>) deselecting the boxes relating to <em>Restore most recently opened project at startup</em> and <em>Restore previously open source documents at startup</em>.</p> <img src="https://remlapmot.github.io/post/2024/macos-rstudio-another/img/rstudio-preference-pane.png" alt="Screenshot of RStudio Desktop preference pane." width="630" style="display: block; margin: auto;"> <img src="https://remlapmot.github.io/post/2024/macos-rstudio-another/img/new-rstudio-window.png" alt="Screenshot of right clicking on the RStudio Desktop icon in the dock." width="504" style="display: block; margin: auto;"> <p>Alternatively, within RStudio Desktop we could select <em>File | Open Project&hellip;</em> but I&rsquo;ve never found that very intuitive and somehow I&rsquo;ve never got used to using RStudio&rsquo;s command palette. If you have several directories configured as RStudio projects (with <em>.Rproj</em> files) that you&rsquo;ve opened before you could open those from the top right project drop down menu or double click them from within Finder.</p> <p>But let&rsquo;s say we want an even more convenient way of doing this &ndash; an icon in the dock. Thanks to <a href="https://stackoverflow.com/a/40673644" target="_blank" rel="noopener">this</a> Stackoverflow answer, it turns out that in a terminal we can issue the following <code>open</code> command to achieve opening a new instance of RStudio Desktop (note RStudio Desktop needs to be installed in your Applications directory for this to work).</p> <pre><code class="language-bash">open -n -a RStudio.app </code></pre> <h2 id="making-an-automator-app-to-open-a-new-instance-of-rstudio-desktop">Making an Automator app to open a new instance of RStudio Desktop</h2> <p>macOS comes with Automator, open that from your Applications directory. Then select <em>File | New</em> and select <em>Application</em> and click <em>Choose</em>. Then from the left most menu select <em>Utilities</em> and double click <em>Run Shell Script</em>. In the main window on the right, leave the shell as zsh and replace the <code>cat</code> text in the box with the <code>open</code> command above.</p> <img src="https://remlapmot.github.io/post/2024/macos-rstudio-another/img/automator-app.png" alt="Screenshot of configuring the automator app to run a shell script to open a new instance of RStudio Desktop." width="992" style="display: block; margin: auto;"> <p>Then press <kbd>Cmd</kbd> + <kbd>S</kbd> or <em>File | Save</em> and save your application in your Applications folder. Give the app a distinct name such as <em>RStudio-new-instance.app</em> or whatever you prefer and then quit Automator.</p> <p>Next we would like our app to have a nice icon.<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup></p> <ul> <li>First download a nice icon from say <a href="https://github.com/amirmasoudabdol/rstudio-icons/tree/main/Icons/macOS/Rectangle" target="_blank" rel="noopener">this</a> repo of logos kindly designed for RStudio (I chose the dark one).</li> <li>Then navigate to the our new app in Finder and press <kbd>Cmd</kbd> + <kbd>I</kbd>.</li> <li>Then drag the <em>.icns</em> file onto the icon in the top left corner of the info box. And tada your app will now use this icon!</li> </ul> <p>And you can even drag the app from Finder onto your dock from where you can simply click the icon to open as many instances of RStudio Desktop as you like.</p> <img src="https://remlapmot.github.io/post/2024/macos-rstudio-another/featured.png" alt="Screenshot of our new app's icon in the dock." width="319" style="display: block; margin: auto;"> <p>And because the new application is in our Applications directory it is found by Alfred (and hopefully Raycast).</p> <img src="https://remlapmot.github.io/post/2024/macos-rstudio-another/img/alfred-launch-rstudio.png" alt="Screenshot of using Alfred to launch RStudio Desktop." width="719" style="display: block; margin: auto;"> <h2 id="summary">Summary</h2> <p>We have created an Automator application which runs a shell script to open a new instance of RStudio Desktop. We have then given this application a new icon and we have placed this on our dock.</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>I follow the approach detailed on <a href="https://www.idownloadblog.com/2023/01/19/how-to-change-app-icons-mac/" target="_blank" rel="noopener">this</a> blog and on the <a href="https://github.com/amirmasoudabdol/rstudio-icons/tree/main?tab=readme-ov-file#usage" target="_blank" rel="noopener">README</a> where I obtained the icon.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div> The MRC IEU ๐Ÿš€ R-Universe of Mendelian randomization related R packages https://remlapmot.github.io/talk/2024_mrcieu-r-universe/ Fri, 26 Jan 2024 14:00:00 +0000 https://remlapmot.github.io/talk/2024_mrcieu-r-universe/ My #GitHubUnwrapped 2023! https://remlapmot.github.io/post/2024/github-unwrapped-2023/ Sun, 07 Jan 2024 00:00:00 +0000 https://remlapmot.github.io/post/2024/github-unwrapped-2023/ <p>My #GitHubUnwrapped 2023!</p> <p align="center"><iframe src="https://drive.google.com/file/d/1bhdY7FPgmYJo_loqIChBJOZ8vdm2mEpA/preview" width="640" height="480" allow="autoplay"></iframe></p> <p>Made with <a href="https://www.githubunwrapped.com/" target="_blank" rel="noopener">https://www.githubunwrapped.com/</a>.</p> Using Git, GitHub Desktop, and GitHub for novice and experienced users https://remlapmot.github.io/talk/2023_code-clinic/ Fri, 01 Dec 2023 11:00:00 +0000 https://remlapmot.github.io/talk/2023_code-clinic/ Fuller reproducibility in Stata ado-files and programs: setting the version and user version https://remlapmot.github.io/post/2023/stata-reproducibility/ Thu, 24 Aug 2023 00:00:00 +0000 https://remlapmot.github.io/post/2023/stata-reproducibility/ <p>Most proficient Stata users have come across the <code>version</code> command. This is an incredibly powerful command, which simply by issuing say</p> <pre><code class="language-stata">version 18 </code></pre> <p>at the top of a do-file or within a program (typically in an ado-file) means that you have pretty much guaranteed your code will run in the same way when you come to run it later (most likely in a newer version of Stata). But it turns out thereโ€™s a subtle difference between issuing <code>version</code> in a do-file/interactively compared to within a program or ado-file.</p> <p>Several years ago I wrote the <strong>reffadjust</strong> package ( <a href="#ref-palmer-sj-2014">Palmer et al. 2014</a>) as part of some work using random effects models ( <a href="#ref-mcdw-sim-2012">Macdonald-Wallis et al. 2012</a>). It has two programs <code>reffadjustsim</code> and <code>reffadjust4nlcom</code> which use the output of various random effects commands, including those from MLwiN, run from Stata using the user-written <code>runmlwin</code> command ( <a href="#ref-leckie-jss-2013">Leckie and Charlton 2013</a>).</p> <p>The <strong>reffadjust</strong> package doesnโ€™t have many users and over the years I hadnโ€™t regularly checked if the programs were still working. But in the ado-files I had set <code>version 13</code>, which gave me some residual confidence that the programs might still work.</p> <p>However, a few years ago, when I eventually did run some test code I saw that the <code>reffadjustsim</code> tests were failing for MLwiN/<code>runmlwin</code> models. I didnโ€™t have time to investigate further at this point, and I didnโ€™t have any intuition whether the error resulted from a change in MLwiN, <code>runmlwin</code>, or Stata.</p> <p>At the beginning of this year one of my colleagues mentioned that they were using <strong>reffadjust</strong> in their work and had observed the same error with <code>reffadjustsim</code>. My guilt kicked in, and I eventually found some time to investigate. I discovered that since I wrote the package, Stata processes matrix row and column stripes (essentially the row and column names) in a more advanced way. This meant that the row and column stripes for covariance elements in the <code>e(V)</code> matrix (the variance-covariance matrix of parameter estimates) from MLwiN/<code>runmlwin</code> models were being renamed when I hadnโ€™t intended them to be, which caused the error.</p> <p>But wait โ€ฆ I had specified <code>version 13</code> at the top of my program, so why was this update in later versions of Stata taking effect?</p> <p>I couldnโ€™t work it out, so I had to ask Stata Technical Support. They were kind enough to tell me that thereโ€™s an additional method of invoking the <code>version</code> command which controls what is known as the โ€œuser versionโ€. There are some modifications in new versions of Stata which are exempt from the basic invocation of the <code>version</code> command (but only in programs and ado-files). In do-files issuing <code>version</code> sets both the version and the user version, however, in programs and ado-files the โ€œuser versionโ€, is set by <code>version #, user</code>, and holds these additional modifications in Stata to the required version.</p> <p>Naturally, this is explained in the <code>version</code> helpfile and <a href="https://www.stata.com/manuals/pversion.pdf" target="_blank" rel="noopener">manual entry</a>, which I admit I had not read until this point. Hence, simply editing the top of my program to</p> <pre><code class="language-stata">version 13 version 13, user </code></pre> <p>fixed my error. So in a program or ado-file, we require both lines, whereas in a do-file weโ€™d only require <code>version 13</code>.</p> <p>We can see the different operation of <code>version</code> by the following short example.</p> <pre><code class="language-stata">/* Do-file/interactive code to set both the version and the user version */ version 13 display c(version), c(userversion), c(stata_version) // 13 13 18.5 </code></pre> <pre><code class="language-stata">/* Ado-file/program code to set both the version and the user version */ program mytest version 13 display c(version), c(userversion), c(stata_version) version 13, user display c(version), c(userversion), c(stata_version) end mytest // 13 18.5 18.5 // 13 13 18.5 </code></pre> <p>In summary, my <code>reffadjustsim</code> command works again for MLwiN/<code>runmlwin</code> models. The updated version is available from <a href="https://github.com/remlapmot/reffadjust" target="_blank" rel="noopener">its GitHub repo</a>. And if you ever need <em>fuller</em> reproducibility in your Stata ado-file or program remember to set both the version and the user version.</p> <h2 id="references">References</h2> <div id="refs" class="references csl-bib-body hanging-indent" entry-spacing="0"> <div id="ref-leckie-jss-2013" class="csl-entry"> <p>Leckie, George, and Chris Charlton. 2013. โ€œ<span class="nocase">runmlwin: A Program to Run the MLwiN Multilevel Modeling Software from within Stata</span>.โ€ <em>Journal of Statistical Software</em> 52 (11): 1โ€“40. <a href="https://doi.org/10.18637/jss.v052.i11" target="_blank" rel="noopener">https://doi.org/10.18637/jss.v052.i11</a>.</p> </div> <div id="ref-mcdw-sim-2012" class="csl-entry"> <p>Macdonald-Wallis, Corrie, Debbie A. Lawlor, Tom Palmer, and Kate Tilling. 2012. โ€œMultivariate Multilevel Spline Models for Parallel Growth Processes: Application to Weight and Mean Arterial Pressure in Pregnancy.โ€ <em>Statistics in Medicine</em> 31 (26): 3147โ€“64. <a href="https://doi.org/doi.org/10.1002/sim.5385" target="_blank" rel="noopener">https://doi.org/doi.org/10.1002/sim.5385</a>.</p> </div> <div id="ref-palmer-sj-2014" class="csl-entry"> <p>Palmer, Tom M., Corrie M. Macdonald-Wallis, Debbie A. Lawlor, and Kate Tilling. 2014. โ€œ<span class="nocase">Estimating adjusted associations between random effects from multilevel models: The reffadjust package</span>.โ€ <em>The Stata Journal</em> 14 (1): 119โ€“40. <a href="https://doi.org/10.1177/1536867X1401400109" target="_blank" rel="noopener">https://doi.org/10.1177/1536867X1401400109</a>.</p> </div> </div> Using allele scores to identify confounding by reverse causation: Studies of alcohol consumption as an exemplar https://remlapmot.github.io/talk/2023_reverse-mr/ Mon, 22 May 2023 13:00:00 +0000 https://remlapmot.github.io/talk/2023_reverse-mr/ My #GitHubUnwrapped 2022! https://remlapmot.github.io/post/2023/github-unwrapped-2022/ Sat, 07 Jan 2023 00:00:00 +0000 https://remlapmot.github.io/post/2023/github-unwrapped-2022/ <p>My #GitHubUnwrapped 2022!</p> <p align="center"><iframe src="https://drive.google.com/file/d/1VVoFQK5aUrgkCppcJXlDMoEPcCKVPW84/preview" width="640" height="480" allow="autoplay"></iframe></p> <p>Made with <a href="https://www.githubunwrapped.com/" target="_blank" rel="noopener">https://www.githubunwrapped.com/</a>.</p> Make your own CRAN-like repository with Linux binary R packages https://remlapmot.github.io/post/2022/make-linux-binary-cran-like-repo/ Wed, 30 Nov 2022 00:00:00 +0000 https://remlapmot.github.io/post/2022/make-linux-binary-cran-like-repo/ <style type="text/css"> .lineheightone { line-height: 1.29; font-variant-ligatures: normal; font-feature-settings: normal; } </style> <h2 id="introduction">Introduction</h2> <p>CRAN is a fantastic resource, in particular because it provides binary packages for Windows and macOS (for both Intel and Apple Silicon Macs). Because there are so many Linux distributions CRAN does not provide binary packages for Linux. Therefore, installing R packages on Linux can be slow because the bundled source packages need to be built on users machines.</p> <p>But let&rsquo;s install a package from the Posit (formerly RStudio) Package Manager on Ubuntu Linux. <img src="https://remlapmot.github.io/post/2022/make-linux-binary-cran-like-repo/img/rstudio-install-tidyverse-rspm-linux-binary.png" alt="Installing binary Linux tidyverse package using Posit Package Manager." style="display: block; margin: auto;"></p> <p><strong><em>Woah!</em></strong> Something magical just happened, we installed a binary R package on Linux! How did that happen, let&rsquo;s find out.<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup></p> <p>Note, obtaining the binary package from the public Posit package manager relies on the R session&rsquo;s <code>HTTPUserAgent</code> option being set. As I understand it, this is set automatically within an RStudio/RStudio Server session. If you are using R in the terminal you can set this with the following code (if you don&rsquo;t have this option set you will obtain the source version of the package).</p> <pre><code class="language-r">options(HTTPUserAgent = sprintf( &quot;R/%s R (%s)&quot;, getRversion(), paste(getRversion(), R.version[&quot;platform&quot;], R.version[&quot;arch&quot;], R.version[&quot;os&quot;]) ) ) </code></pre> <h2 id="building-bundled-source-and-binary-packages">Building bundled source and binary packages</h2> <p>I will use an example of one of my own packages <strong>OneSampleMR</strong>. I am running RStudio server on Ubuntu Linux, Focal Fossa through Windows Subsystem for Linux.</p> <p>The package sources are in a Git repository hosted on GitHub, <a href="https://github.com/remlapmot/OneSampleMR" target="_blank" rel="noopener">here</a>. There is an <code>.Rproj</code> file, which we open in RStudio as a project.</p> <p>To build an R package we require all of its dependency packages are installed, so we install those with <code>devtools::install_dev_deps()</code> and if your package requires any system libraries those must be installed too.</p> <p>The Build pane gives us two convenient options, which will build either the bundled source package or binary package through calls to <code>devtools::build()</code>. <img src="https://remlapmot.github.io/post/2022/make-linux-binary-cran-like-repo/img/rstudio-build-pane-build-options.png" alt="Screenshot of build pane options for building bundled source and binary packages." style="display: block; margin: auto;"></p> <p>Clicking on both in turn we see the following. <img src="https://remlapmot.github.io/post/2022/make-linux-binary-cran-like-repo/img/rstudio-build-source-package.png" alt="Screenshot of building a bundled source package in RStudio." style="display: block; margin: auto;"></p> <img src="https://remlapmot.github.io/post/2022/make-linux-binary-cran-like-repo/img/rstudio-build-binary-package.png" alt="Screenshot of building a binary package in RStudio." style="display: block; margin: auto;"> <p>The bundled source package has been built as <code>OneSampleMR_0.1.2.tar.gz</code> and the binary package has been built as <code>OneSampleMR_0.1.2_R_x86_64-pc-linux-gnu.tar.gz</code>. Both files are in the directory above the project.</p> <p>We can achieve the same output by making direct calls to <code>R CMD build</code> and <code>R CMD install --build</code> in a shell if preferred. We can test that these install as follows.</p> <img src="https://remlapmot.github.io/post/2022/make-linux-binary-cran-like-repo/img/rstudio-install-source-package.png" alt="Screenshot of installing a bundled source package in RStudio." style="display: block; margin: auto;"> <img src="https://remlapmot.github.io/post/2022/make-linux-binary-cran-like-repo/img/rstudio-install-binary-package.png" alt="Screenshot of installing a binary package in RStudio." style="display: block; margin: auto;"> <h2 id="cran-structure-for-bundled-source-package-files">CRAN structure for bundled source package files</h2> <p>In two excellent blog posts Marks Sellors describes how to make a CRAN-like repository.<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup></p> <p>To host bundled source packages, such as our <code>OneSampleMR_0.1.2.tar.gz</code> file, we require the following directory structure (noting that the <code>latest</code> directory is optional, but allows us to add snapshot directories if we wanted to).</p> <!-- # brew install tree --> <!-- tree ~/Documents/GitHub/cran/site -d -I '__linux__|bin' --noreport --> <pre><code class="language-plaintext{.lineheightone}">/mycran โ””โ”€โ”€ latest โ””โ”€โ”€ src โ””โ”€โ”€ contrib โ””โ”€โ”€ 4.3.0 โ””โ”€โ”€ Recommended </code></pre> <p>Specifically, we place the <code>{package}_{version}.tar.gz</code> files into the <code>.../src/contrib</code> directory. In that directory we then run</p> <pre><code class="language-r">tools::write_PACKAGES(type = &quot;source&quot;) </code></pre> <p>which generates 3 additional files (<code>PACKAGES</code>, <code>PACKAGES.gz</code>, and <code>PACKAGES.rds</code>) which R will use to query what packages are available in our repository when its served on the web.</p> <h2 id="adding-windows-and-macos-binary-r-packages">Adding Windows and macOS binary R packages</h2> <p>We saw above how to build a binary Linux package. The same process, when repeated on Windows will generate a file called <code>{package}_{version}.zip</code> and <code>{package}_{version}.tgz</code> on macOS (on Macs with both Intel and Apple Silicon processors).</p> <p>Assuming that we have some of these files we need to know where to put them. Since CRAN distributes binary packages for Windows and macOS we follow their directory structure, which is as follows.</p> <!-- brew install tree --> <!-- tree ~/Documents/GitHub/cran/site -d -I '__linux__' --noreport --> <pre><code class="language-plaintext{.lineheightone}">/mycran โ””โ”€โ”€ latest โ”œโ”€โ”€ bin โ”‚ โ”œโ”€โ”€ macosx โ”‚ โ”‚ โ”œโ”€โ”€ big-sur-arm64 โ”‚ โ”‚ โ”‚ โ””โ”€โ”€ contrib โ”‚ โ”‚ โ”‚ โ””โ”€โ”€ 4.2 โ”‚ โ”‚ โ””โ”€โ”€ contrib โ”‚ โ”‚ โ””โ”€โ”€ 4.2 โ”‚ โ””โ”€โ”€ windows โ”‚ โ””โ”€โ”€ contrib โ”‚ โ””โ”€โ”€ 4.2 โ””โ”€โ”€ src โ””โ”€โ”€ contrib โ””โ”€โ”€ 4.3.0 โ””โ”€โ”€ Recommended </code></pre> <p>Noting that the current version of R is 4.2.2 and that the relevant directory name with the minor version number changes when R&rsquo;s current minor version number changes, we place</p> <ul> <li>macOS arm64 binary packages (for Macs with Apple Silicon processors) into the <code>.../bin/macosx/big-sur-arm64/contrib/4.2/</code> directory</li> <li>macOS x86_64 binary packages (for Macs with Intel processors) into the <code>.../bin/macosx/contrib/4.2/</code> directory, and</li> <li>Windows binary packages into the <code>.../bin/windows/contrib/4.2/</code> directory.</li> </ul> <p>We then run <code>tools::write_PACKAGES(type = &quot;mac.binary&quot;)</code> (changing to <code>type = &quot;win.binary&quot;</code> as required) in each of these directories to generate the 3 <code>PACKAGES</code> files.</p> <p>We can confirm this directory structure using the <code>contrib.url()</code> function (the last command below was run on an Apple Silicon Mac).</p> <pre><code class="language-r">contrib.url(&quot;&quot;, type = &quot;source&quot;) ## [1] &quot;/src/contrib&quot; contrib.url(&quot;&quot;, type = &quot;win.binary&quot;) ## [1] &quot;/bin/windows/contrib/4.2&quot; contrib.url(&quot;&quot;, type = &quot;mac.binary&quot;) ## [1] &quot;/bin/macosx/contrib/4.2&quot; contrib.url(&quot;&quot;, type = &quot;binary&quot;) ## [1] &quot;/bin/macosx/big-sur-arm64/contrib/4.2&quot; </code></pre> <p>And they are also listed in the R Installation and Administration manual.<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup></p> <p>On Apple Silicon Macs the <code>big-sur-arm64</code> filepath corresponds to the end of <code>.Platform$pkgType</code>.</p> <pre><code class="language-r">.Platform$pkgType ## [1] &quot;mac.binary.big-sur-arm64&quot; </code></pre> <h2 id="where-to-store-and-how-to-name-linux-binary-r-packages">Where to store and how to name Linux binary R packages?</h2> <p>CRAN does not distribute Linux binary packages and so there is no directory structure from them to copy.</p> <p>However both the Posit Package Manager and the <a href="https://r4pi.org/" target="_blank" rel="noopener">R4Pi</a> project achieve this in a very clever way.<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup></p> <p>We saw above that on Linux bundled source packages have filenames of the form <code>{package}_{version}.tar.gz</code> whereas the binary package filenames are of the form <code>{package}_{version}_R_x86_64-pc-linux-gnu.tar.gz</code> (the text after <code>{package}_{version}_</code> may be different depending on your machine and distro).</p> <p>To distribute the Linux binary packages we create a parallel directory structure, which takes the same form as for the bundled source packages. In the case of Ubuntu Focal Fossa the Posit Package Manager uses <code>__linux__/focal/latest/src/contrib</code>.</p> <p>They then rename the <code>{package}_{version}_R_x86_64-pc-linux-gnu.tar.gz</code> files to the same form as the bundled source package files, i.e., to <code>{package}_{version}.tar.gz</code>. And they put them in this new <code>src/contrib</code> directory.</p> <p>The structure of our CRAN-like repository (well in fact our 2 parallel repositories) is now.</p> <!-- tree ~/Documents/GitHub/cran/site -d -I 'jammy' --noreport --> <pre><code class="language-plaintext{.lineheightone}">/mycran โ”œโ”€โ”€ __linux__ โ”‚ โ””โ”€โ”€ focal โ”‚ โ””โ”€โ”€ latest โ”‚ โ””โ”€โ”€ src โ”‚ โ””โ”€โ”€ contrib โ”‚ โ””โ”€โ”€ 4.3.0 โ”‚ โ””โ”€โ”€ Recommended โ””โ”€โ”€ latest โ”œโ”€โ”€ bin โ”‚ โ”œโ”€โ”€ macosx โ”‚ โ”‚ โ”œโ”€โ”€ big-sur-arm64 โ”‚ โ”‚ โ”‚ โ””โ”€โ”€ contrib โ”‚ โ”‚ โ”‚ โ””โ”€โ”€ 4.2 โ”‚ โ”‚ โ””โ”€โ”€ contrib โ”‚ โ”‚ โ””โ”€โ”€ 4.2 โ”‚ โ””โ”€โ”€ windows โ”‚ โ””โ”€โ”€ contrib โ”‚ โ””โ”€โ”€ 4.2 โ””โ”€โ”€ src โ””โ”€โ”€ contrib โ””โ”€โ”€ 4.3.0 โ””โ”€โ”€ Recommended </code></pre> <p>We then run</p> <pre><code class="language-r">tools::write_PACKAGES(type = &quot;source&quot;) </code></pre> <p>in the <code>.../__linux__/distro-name/src/contrib</code> directory for each Linux distribution to generate the <code>PACKAGES</code> files.</p> <p>Once this directory structure is served we can set our repository in R to <code>.../__linux__/focal/latest</code> and R will find the binary package <code>{package}_{version}.tar.gz</code> files in the <code>...__linux__/focal/src/contrib/</code> directory, as per the first figure in this post.</p> <p>Note that the <code>{package}_{version}.tar.gz</code> files within <code>__linux__/focal/latest/src/contrib</code> do not all have to be of binary packages. They can be a mix of bundled source and binary packages, which is helpful if you haven&rsquo;t had time to build all your binary package files.</p> <p>We can also confirm that on Ubuntu Focal Fossa</p> <pre><code class="language-r">.Platform$pkgType ## [1] &quot;source&quot; </code></pre> <p>therefore, on Linux, <code>utils:::resolvePkgType()</code> always returns <code>&quot;source&quot;</code>. Hence, on Linux, <code>contrib.url()</code> always returns <code>/src/contrib</code> regardless of its <code>type</code> argument.</p> <h2 id="testing-your-cran-like-repositories-locally">Testing your CRAN-like repositories locally</h2> <p>You can either run a local web server or use the <code>file://...</code> URL notation as your <code>repos</code> global options setting (set with <code>option(repos = ...)</code> or as the <code>repos</code> argument to <code>install.packages()</code>).</p> <p>Here&rsquo;s a screenshot of installing a binary package on Ubuntu Focal Fossa. <img src="https://remlapmot.github.io/post/2022/make-linux-binary-cran-like-repo/img/cran-like-focal-example.png" alt="Screenshot of testing a CRAN-like repository on Ubuntu Focal Fossa." style="display: block; margin: auto;"></p> <h2 id="summary">Summary</h2> <p>We have taken a look at the structure of a CRAN-like repository and built bundled source and binary package files. We saw that the trick for distributing Linux binary packages is to make a parallel directory with the same structure as that required for bundled source packages and that we need to rename the binary package files to have the same filename format as the bundled source package files.</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>Note that there are other approaches to distributing binary R packages on Linux, see <a href="https://cran.r-project.org/bin/linux/" target="_blank" rel="noopener">https://cran.r-project.org/bin/linux/</a> and links therein, <a href="https://eddelbuettel.github.io/r2u/" target="_blank" rel="noopener">https://eddelbuettel.github.io/r2u/</a>, and <a href="https://enchufa2.github.io/bspm/" target="_blank" rel="noopener">https://enchufa2.github.io/bspm/</a>.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p> <a href="https://blog.sellorm.com/2019/03/29/lifting-the-lid-on-cran/" target="_blank" rel="noopener">https://blog.sellorm.com/2019/03/29/lifting-the-lid-on-cran/</a> and <a href="https://blog.sellorm.com/2019/03/30/build-your-own-cran-like-repo/" target="_blank" rel="noopener">https://blog.sellorm.com/2019/03/30/build-your-own-cran-like-repo/</a>] There is also the <a href="https://cran.r-project.org/package=miniCRAN" target="_blank" rel="noopener"><strong>miniCRAN</strong></a> package to help do this, but we don&rsquo;t need to use this for the following explanation.&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3"> <p>See <a href="https://cran.r-project.org/doc/manuals/R-admin.html#Setting-up-a-package-repository" target="_blank" rel="noopener">https://cran.r-project.org/doc/manuals/R-admin.html#Setting-up-a-package-repository</a>.&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:4"> <p>See <a href="https://packagemanager.rstudio.com/client/#/repos/2/overview" target="_blank" rel="noopener">https://packagemanager.rstudio.com/client/#/repos/2/overview</a> and <a href="https://pkgs.r4pi.org/" target="_blank" rel="noopener">https://pkgs.r4pi.org/</a>&#160;<a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div> A review of group-based methods for teaching statistics in higher education https://remlapmot.github.io/talk/2022_horizons-in-stem/ Wed, 29 Jun 2022 16:20:00 +0000 https://remlapmot.github.io/talk/2022_horizons-in-stem/ Investigating and fixing a new CRAN check error in my OneSampleMR package https://remlapmot.github.io/talk/2022_check-error/ Fri, 20 May 2022 11:30:00 +0000 https://remlapmot.github.io/talk/2022_check-error/ R and Stata packages for one-sample Mendelian randomization analyses: OneSampleMR and ivonesamplemr https://remlapmot.github.io/talk/2022_stats-discussion/ Thu, 20 Jan 2022 13:00:00 +0000 https://remlapmot.github.io/talk/2022_stats-discussion/ Parameter collapsibility and noncollapsibility in statistical models: what you need to know https://remlapmot.github.io/talk/2021_stats-discussion/ Mon, 18 Jan 2021 13:00:00 +0000 https://remlapmot.github.io/talk/2021_stats-discussion/ bpbounds: R package and web app https://remlapmot.github.io/talk/2019_mr/ Wed, 17 Jul 2019 00:00:00 +0000 https://remlapmot.github.io/talk/2019_mr/ Directed acyclic graphs: what are they and what are they useful for? https://remlapmot.github.io/talk/2018_keele/ Tue, 04 Dec 2018 12:00:00 +0000 https://remlapmot.github.io/talk/2018_keele/ mrrobust: A Stata package for MR-Egger regression type analyses https://remlapmot.github.io/talk/2017_stata/ Fri, 08 Sep 2017 00:00:00 +0000 https://remlapmot.github.io/talk/2017_stata/ mrrobust: A Stata package implementing MR-Egger regression type analyses https://remlapmot.github.io/talk/2017_mr/ Wed, 12 Jul 2017 13:00:00 +0000 https://remlapmot.github.io/talk/2017_mr/ Corrected standard errors for two-stage residual inclusion estimators and a Stata package for MR-Egger regression type analyses https://remlapmot.github.io/talk/2017_leibniz/ Thu, 20 Apr 2017 13:00:00 +0000 https://remlapmot.github.io/talk/2017_leibniz/ Some topics from Mendelian randomization https://remlapmot.github.io/talk/2017_lstm/ Tue, 07 Mar 2017 12:00:00 +0000 https://remlapmot.github.io/talk/2017_lstm/ Meta-analytic structural equation modelling: Application to Mendelian randomization studies https://remlapmot.github.io/talk/2016_ukcim/ Thu, 14 Apr 2016 12:00:00 +0000 https://remlapmot.github.io/talk/2016_ukcim/ Fitting fixed and random effects meta-analysis models using structural equation models https://remlapmot.github.io/talk/2015_iscb/ Thu, 27 Aug 2015 13:00:00 +0000 https://remlapmot.github.io/talk/2015_iscb/ Corrections to Probit and logistic control function estimator standard errors for marginal parameters https://remlapmot.github.io/talk/2015_mr/ Mon, 22 Jun 2015 13:00:00 +0000 https://remlapmot.github.io/talk/2015_mr/ Lack of identification in structural mean models and multiple paired comparisons for investigating pleiotropy https://remlapmot.github.io/talk/2014_ieu/ Tue, 09 Sep 2014 12:00:00 +0000 https://remlapmot.github.io/talk/2014_ieu/ Generalised method of moments estimation of mediation models and structural mean models https://remlapmot.github.io/talk/2014_bk/ Wed, 12 Mar 2014 13:00:00 +0000 https://remlapmot.github.io/talk/2014_bk/ Topics in instrumental variable estimation: structural mean models and bounds https://remlapmot.github.io/talk/2013_rss/ Thu, 07 Nov 2013 16:00:00 +0000 https://remlapmot.github.io/talk/2013_rss/ Strengthening Mendelian randomization through utilizing multiple independent paired combinations of genetic variants to evaluate potential pleiotropy https://remlapmot.github.io/talk/2013_iscb/ Mon, 26 Aug 2013 13:00:00 +0000 https://remlapmot.github.io/talk/2013_iscb/ Topics in instrumental variable estimation: structural mean models and bounds https://remlapmot.github.io/talk/2013_crism/ Thu, 30 May 2013 17:00:00 +0000 https://remlapmot.github.io/talk/2013_crism/ Generalised method of moments estimation of mediation models and structural mean models https://remlapmot.github.io/talk/2013_ukcim/ Wed, 15 May 2013 09:00:00 +0000 https://remlapmot.github.io/talk/2013_ukcim/ Topics in instrumental variable estimation: structural mean models and bounds https://remlapmot.github.io/talk/2012_gottingen/ Wed, 01 Feb 2012 13:00:00 +0000 https://remlapmot.github.io/talk/2012_gottingen/ Generalised method of moments estimation of structural mean models https://remlapmot.github.io/talk/2011_stata/ Thu, 15 Sep 2011 00:00:00 +0000 https://remlapmot.github.io/talk/2011_stata/ Estimation using structural mean models with multiple instruments https://remlapmot.github.io/talk/2011_lshtm/ Wed, 14 Sep 2011 13:00:00 +0000 https://remlapmot.github.io/talk/2011_lshtm/ Generalised method of moments estimation of structural mean models https://remlapmot.github.io/talk/2011_iscb/ Tue, 23 Aug 2011 13:00:00 +0000 https://remlapmot.github.io/talk/2011_iscb/ Estimation of structural mean models with multiple instruments https://remlapmot.github.io/talk/2011_rss/ Thu, 26 May 2011 13:00:00 +0000 https://remlapmot.github.io/talk/2011_rss/ Mendelian randomization: using genotypes as instrumental variables in epidemiological studies https://remlapmot.github.io/talk/2011_rss-avon/ Thu, 10 Feb 2011 16:00:00 +0000 https://remlapmot.github.io/talk/2011_rss-avon/ Using multiple independent combinations of genetic variants to strengthen causal inference in Mendelian randomization studies: height and lung function as an example https://remlapmot.github.io/talk/2010_rss/ Wed, 15 Sep 2010 13:00:00 +0000 https://remlapmot.github.io/talk/2010_rss/ Instrumental variable estimation of the causal risk ratio in cohorts https://remlapmot.github.io/talk/2010_iscb/ Mon, 30 Aug 2010 13:00:00 +0000 https://remlapmot.github.io/talk/2010_iscb/ Including multiple instrumental variables in Mendelian randomization analyses https://remlapmot.github.io/talk/2009_ghent/ Sat, 14 Nov 2009 12:00:00 +0000 https://remlapmot.github.io/talk/2009_ghent/ Contour enhanced funnel plots for meta-analysis https://remlapmot.github.io/talk/2009_stata/ Fri, 11 Sep 2009 13:00:00 +0000 https://remlapmot.github.io/talk/2009_stata/ Including multiple instrumental variables in Mendelian randomization analyses https://remlapmot.github.io/talk/2009_iscb/ Wed, 26 Aug 2009 13:00:00 +0000 https://remlapmot.github.io/talk/2009_iscb/ Performing Bayesian analysis in Stata using WinBUGS https://remlapmot.github.io/talk/2007_stata/ Mon, 10 Sep 2007 13:00:00 +0000 https://remlapmot.github.io/talk/2007_stata/ An adjusted instrumental-variable model for Mendelian randomization https://remlapmot.github.io/talk/2007_iges/ Sat, 08 Sep 2007 00:00:00 +0000 https://remlapmot.github.io/talk/2007_iges/ Meta-analysis of Mendelian randomization studies https://remlapmot.github.io/talk/2007_iscb/ Wed, 01 Aug 2007 13:00:00 +0000 https://remlapmot.github.io/talk/2007_iscb/ Meta-analysis of Mendelian randomization studies https://remlapmot.github.io/talk/2007_ysm/ Thu, 12 Apr 2007 13:00:00 +0000 https://remlapmot.github.io/talk/2007_ysm/ Incorporating measures of study similarity in a meta-analysis https://remlapmot.github.io/talk/2006_iscb/ Wed, 30 Aug 2006 17:00:00 +0000 https://remlapmot.github.io/talk/2006_iscb/