Type Checker</th>	Total Test Case Passes</th>	Total Test Case Partial</th>	Total False Positives</th>	Total False Negatives</th></tr></thead>
zuban 0.0.20</td>	97</td>	42</td>	152</td>	89</td></tr>
ty 0.0.1-alpha.19 (e9cb838b3 2025-08-19)</td>	20</td>	119</td>	371</td>	603</td></tr>
Local:ty ruff/0.12.11+27 (0bf5d2a20 2025-08-29)</td>	20</td>	119</td>	370</td>	590</td></tr>
pyrefly 0.30.0</td>	81</td>	58</td>	100</td>	187</td></tr> </tbody></table> Review of progress</h2> `zuban</code> has a lead, having full Pass</code> on ~69% of test cases, compared with ~15% for ty</code> and ~58% for pyrefly</code>. Which makes sense as though it's released in a similar time period to ty</code> and pyrefly</code> it has been in active development in private for several years.</p>` `The thing that surprised me more was how much progress pyrefly</code> has made when compared to ty</code>. Both broke cover and released their first advertised alpha builds around the same time, in the run up to PyCon 2025. This can maybe be partially explained from a point raised in Edward Li's excellent blog post on the Typing Summit at PyCon 2025</a> which mentions that the pyrefly</code> team devoted a lot of up-front time to solving some of the hard problems, such as generics.</p>` Relevance?</h2> I initially came across the conformance test suite because ty</code> runs every PR against the test suite and diffs the PR results against the results from main</code> to ensure changes are desired. From this it's become a surprisingly useful learning tool for some of the more advanced typing topics, but the advanced nature of these topics raises an important question: how relevant is the conformance suite pass rate for the average Python developer?</strong></p> The Gap Between Conformance and Real-World Usage</h3> The conformance test suite focuses heavily on advanced typing features that, while important for the specification, may not reflect the day-to-day typing needs of most Python codebases. Many of the test cases cover complex scenarios involving:</p> Advanced generic variance and bounds</li> Complex protocol inheritance hierarchies</li> Edge cases in structural subtyping</li> Intricate interactions between multiple typing features</li> </ul> In contrast, the majority of Python codebases primarily use:</p> Basic type annotations (str</code>, int</code>, List[str]</code>, etc.)</li> Simple class hierarchies</li> Optional types and Union types</li> Basic generic containers</li> </ul> Practical Experience vs. Test Scores</h3> My experience using the ty</code> VSCode extension in place of Pylance</code> across various projects and libraries tells a different story than the conformance test scores suggest. Despite ty</code>'s relatively low 15% full pass rate, it has been surprisingly effective at catching real bugs and providing useful feedback for common typing patterns.</p> This suggests that while conformance test coverage is important for specification compliance and handling edge cases, it may not be the best predictor of a type checker's utility for everyday Python development. The features that matter most for typical codebases appear to be working well across all three tools, even when they struggle with more esoteric typing scenarios.</p> What This Means for Adoption Decisions</h3> For teams evaluating these type checkers, the conformance scores provide valuable insight into specification compliance, but shouldn't be the sole deciding factor. Consider:</p> For greenfield projects</strong>: Any of these tools will likely handle your immediate needs well</li> For large, complex codebases or libraries leaning on more esoteric generic patterns</strong>: Higher conformance scores may indicate better handling of advanced patterns you might encounter</li> For teams new to typing</strong>: The difference in conformance scores may be less relevant than IDE integration, error message quality, and performance</li> </ul> Other resources to learn more</h1> Edward Li's excellent blog post comparing ty</code> and pyrefly</code></a> which also contains the videos recorded at the PyCon typing summit which both ty</code> and pyrefly</code> gave presentations at.</li> Happy Path Programming - 114 ty: Fast Python Type Checking with Carl Meyer</a></li> Happy Path Programming - 115 More Python Type Checking! Pyrefly with Aaron Pollack & Steven Troxler</a></li> Talk Python - ty: Astral's New Type Checker (Formerly Red-Knot)</a></li> </ul> Footnotes</h1> ^1</sup>Which can be demonstrated in the open issues</a> on ruff tagged with type-inference</code> which are bugs or new features that can only be resolved with ruff</code> having access to deeper type inference data that ty</code> can supply.</p> </div> ^2</sup>David has indicated a plan to make source available in the future</a> when adding Zuban to the Python typing conformance suite.</p> </div> ^{3</sup> Full pricing information at: https://zubanls.com/pricing/</del> Update (September 2025):</strong> Pricing is now available on request for the non-AGPL license.</p> </div>}^{4</sup> This is just for this blog post, no plans to seek merging this.</p> </div>} TIL: Flowistry tool to understand information flow in Rust Unknown — Thu, 28 Aug 2025 00:00:00 +0000 When watching the excellent talk Rust for Everyone</a> by Will Crichton the item that stood out as potentially most useful day to day was Flowistry</a>. It is a VSCode extension, when a user clicks a variable the extension greys out all code that the variable does not interact with, it seems like it could be very useful when debugging.</p> About Unknown — Wed, 27 Aug 2025 00:00:00 +0000 I am Rob Hand, a Staff Software Engineer currently working on Personalisation at M&S. Enjoy learning new things and using those things to solve more complex problems.</p> For an (hopefully) more up to date snapshot see Now</a> page.</p> You can find me on GitHub</a> or send me a message on Bluesky</a>. My CV is available here</a>.</p> Boardgames Unknown — Wed, 27 Aug 2025 00:00:00 +0000 Now Unknown — Wed, 27 Aug 2025 00:00:00 +0000 (This is a now page</a>, and if you have your own site, you should make one</a>, too.)</strong></p> reading: Children of Memory - Adrian Tchaikovsky</li> watching: Nothing regular since Andor Season 2, looking forward to season 2 of Fallout</li> learning: Rust🦀</a> and Gleam⭐</a>.</li> writing: Attempting to build the habit of Journaling with the help of LogSeq</a></li> listening: See on Last.fm</a></li> playing: Mainly Roguelike games that require less time commitment such as: Astral Ascent</a> or Risk of Rain 2</a></li> </ul> Continuing to try and spend more time on hobbies instead of doomsrolling or watching too much YouTube once the work day is over and the kids are put to bed. Not always</del> Rarely</strong> succeeding.</p> Current Projects:</p> loxide</a> - Implementing an interpreter for lox language from https://craftinginterpreters.com/ in Rust. At the point of adding support for user defined functions.</li> z2p-axum</a> - Following along with the Zero to Prod book but re-implementing the application in Axum instead of Actix.</li> </ul> Finished:</p> chip8</a> - A Chip-8</a> interpreter written in no-std Rust.</li> gridlife</a> - A library for Conways Game of Life in Rust with a TUI to simulate random configurations.</li> pngme</a> - Implementation of https://jrdngr.github.io/pngme_book/ which became my go to testing/learning project. Expanded the original with: GUI, Python library binding and crate splitting the project.</li> </ul> On Hold:</p> snake</a> - A snake clone TUI built using Ratatui</li> </ul> Last updated on 27th August 2025.</p> TIL: Now page Unknown — Wed, 27 Aug 2025 00:00:00 +0000 Added a Now</a> page to blog based on nownownow</a>. Since I don't post on social media this seemed like a nice way to share what I am up to at the moment (to my many many readers... crickets</strong></em>)</p> Bridging Python & Rust: A Walkthrough of using Py03 Unknown — Sun, 18 May 2025 00:00:00 +0000 Bridging Python and Rust: A Practical Guide with PyO3</a></li> Project Structure</a></li> Step 1: Exposing Rust Functions with PyO3</a></li> Step 2: Building and Packaging</a></li> Step 3: Using the Library from Python</a></li> Step 4: Handling Errors</a></li> Wrapping Up</a></li> </ul> Bridging Python and Rust: A Practical Guide with PyO3</h1> Sometimes Python just isn't fast enough, or you want to reuse some Rust code without rewriting it. PyO3</a> makes it surprisingly easy to call Rust from Python (or less commonly vice-versa). Here’s how I created pngme-python</a> crate, to expose my already existing pngme</a> Rust crate as a python library.</p> Project Structure</h2> pngme/ </span>├── src/ </span>│ └── lib.rs </span>├── Cargo.toml </span>pngme-python/ </span>├── src/ </span>│ └── lib.rs </span>├── tests/ </span>│ └── test_pngme.py </span>├── Cargo.toml </span>├── pyproject.toml </span>├── README.md </span> </span></code></pre> pngme/src/lib.rs</strong>: The original Rust code with PNG manipulation functionality, that we want to expose as a Python library</li> pngme-python/src/lib.rs</strong>: The Py03 bindings.</li> test_pngme.py</strong>: Python tests to verify the bindings work correctly</li> pngme-python/src/Cargo.toml</strong>: The build configuration for the Rust portion of the Py03/Maturin build process.</li> pyproject.toml</strong>: Python packaging configuration for Maturin</li> </ul> Step 1: Exposing Rust Functions with PyO3</h2> PyO3 lets you turn Rust functions into Python-callable methods with minimal fuss. Here’s a trimmed-down version of the encode</code>, decode</code>, and remove</code> functions from src/lib.rs</a>:</p> </span>use </span>pyo3::prelude::</span></span>; </span> </span>#[pymodule] </span>#[pyo3(name </span>= </span>"pngme"</span>)] </span>mod </span>pngme_python { </span> </span>use </span>pyo3::exceptions::{PyFileNotFoundError, PyIOError, PyValueError}; </span> </span>use </span>pyo3::{prelude::</span></span>, PyResult}; </span> </span>use </span>std::path::PathBuf; </span> </span> </span>use </span>pngme_lib::{decode </span>as</span> png_decode, encode </span>as</span> png_encode, remove </span>as</span> png_remove, Error}; </span> </span> #[pyfunction] </span> </span>pub fn </span>encode</span>(</span>path</span>: PathBuf, </span>chunk_type</span>: String, </span>message</span>: String) -> PyResult<()> { </span> </span>let</span> result </span>= </span>png_encode</span>(</span>&</span>path, </span>&</span>chunk_type, message); </span> </span>match</span> result { </span> </span>Ok</span>(</span>_</span>) </span>=> </span>Ok</span>(()), </span> </span>Err</span>(e) </span>=> </span>match</span> e { </span> Error::FileNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyFileNotFoundError::new_err(e.</span>to_string</span>())), </span> Error::Read { source: s } </span>=> </span>Err</span>(PyIOError::new_err(s.</span>to_string</span>())), </span> Error::PNGParse </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> Error::InvalidChunkType { source: s, </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(s.</span>to_string</span>())), </span> Error::PNGWrite { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> Error::ChunkNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> Error::StrConversion </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> }, </span> } </span> } </span> </span> #[pyfunction] </span> </span>pub fn </span>decode</span>(</span>path</span>: PathBuf, </span>chunk_type</span>: String) -> PyResult<</span>String</span>> { </span> </span>let</span> result </span>= </span>png_decode</span>(</span>&</span>path, </span>&</span>chunk_type); </span> </span>match</span> result { </span> </span>Ok</span>(msg) </span>=> </span>Ok</span>(msg), </span> </span>Err</span>(e) </span>=> </span>match</span> e { </span> Error::FileNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyFileNotFoundError::new_err(e.</span>to_string</span>())), </span> Error::Read { </span>.. </span>} </span>=> </span>Err</span>(PyIOError::new_err(e.</span>to_string</span>())), </span> Error::PNGParse </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> Error::InvalidChunkType { source: s, </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(s.</span>to_string</span>())), </span> Error::PNGWrite { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> Error::ChunkNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> Error::StrConversion </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> }, </span> } </span> } </span> </span> #[pyfunction] </span> </span>pub fn </span>remove</span>(</span>path</span>: PathBuf, </span>chunk_type</span>: String) -> PyResult<()> { </span> </span>let</span> result </span>= </span>png_remove</span>(</span>&</span>path, </span>&</span>chunk_type); </span> </span>match</span> result { </span> </span>Ok</span>(</span>_</span>) </span>=> </span>Ok</span>(()), </span> </span>Err</span>(e) </span>=> </span>match</span> e { </span> Error::FileNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyFileNotFoundError::new_err(e.</span>to_string</span>())), </span> Error::Read { </span>.. </span>} </span>=> </span>Err</span>(PyIOError::new_err(e.</span>to_string</span>())), </span> Error::PNGParse </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> Error::InvalidChunkType { source: s, </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(s.</span>to_string</span>())), </span> Error::PNGWrite { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> Error::ChunkNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> Error::StrConversion </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())), </span> }, </span> } </span> } </span>} </span></code></pre> The key parts of this implementation:</p> The #[pymodule]</code> macro creates a Python module from the Rust module it encloses.</li> Each #[pyfunction]</code> within the Rust module adds a Rust function to Python the #[pymodule]</code>.</li> Each Rust error type must be handled and mapped to the appropriate Python exceptions.</li> </ul> Step 2: Building and Packaging</h2> Maturin</a> handles compiling the Rust code and packaging it as a Python wheel. Two configuration files control this process:</p> Cargo.toml</strong></p> [package] </span>name </span>= </span>"pngme-python" </span>version </span>= </span>"0.1.0" </span>edition </span>= </span>"2021" </span> </span>[lib] </span>name </span>= </span>"pngme" </span>crate-type </span>= [</span>"cdylib"</span>] </span> </span>[dependencies] </span>pyo3 </span>= { </span>version </span>= </span>"0.18.3"</span>, </span>features </span>= [</span>"extension-module"</span>] } </span>pngme-lib </span>= { </span>path </span>= </span>"../pngme" </span>} </span></code></pre> pyproject.toml</strong></p> [build-system] </span>requires </span>= [</span>"maturin>=1.0,<2.0"</span>] </span>build-backend </span>= </span>"maturin" </span> </span>[project] </span>name </span>= </span>"pngme" </span>version </span>= </span>"0.1.0" </span>description </span>= </span>"Python bindings for pngme" </span>readme </span>= </span>"README.md" </span> </span>[tool.maturin] </span>features </span>= [</span>"pyo3/extension-module"</span>] </span></code></pre> Building is as simple as:</p> </span>maturin develop </span></code></pre> This creates a Python wheel that you can use directly or publish to PyPI.</p> Step 3: Using the Library from Python</h2> Once built, and installed, just import and use the module in Python. An example can be found in tests/test_pngme.py</a>:</p> </span>import </span>pngme </span> </span>def test_pngme_encode</span>(): </span> file_location </span>= </span>"./crates/pngme-python/tests/dice.png" </span> pngme.encode(file_location, </span>"ruSt"</span>, </span>"some message"</span>) </span> output </span>= </span>pngme.decode(file_location, </span>"ruSt"</span>) </span> </span>assert </span>output </span>== </span>"some message" </span> pngme.remove(file_location, </span>"ruSt"</span>) </span> nothing </span>= </span>pngme.decode(file_location, </span>"ruSt"</span>) </span> </span>assert </span>nothing </span>== </span>"No secret message found" </span></code></pre> Step 4: Handling Errors</h2> PyO3 lets you map Rust errors to Python exceptions, so Python users get idiomatic error messages as shown below:</p> </span>import </span>pytest </span> </span>def test_pngme_unknown_file</span>(): </span> </span>with </span>pytest.raises(</span>FileNotFoundError</span>) </span>as </span>exc: </span> pngme.encode(</span>"unknown.png"</span>, </span>"ruSt"</span>, </span>"some message"</span>) </span> </span>assert </span>'File not found "unknown.png"' </span>in </span>str</span>(exc.value) </span></code></pre> Wrapping Up</h2> PyO3 makes it easy to bring Rust’s speed and safety to Python, with natural error handling and a smooth workflow. If you want to squeeze more performance out of Python or reuse Rust code, give it a try.</p> References:</strong></p> PyO3 Documentation</a></li> Maturin Documentation</a></li> pngme-python Source Code</a></li> </ul> Project-Based Learning: The Method That Made Rust Finally Click Unknown — Fri, 07 Feb 2025 00:00:00 +0000 Why?</a></li> Previous Learning Attempts</a></li> Project based learning</a> PNGme</a></li> CodeCrafters</a></li> </ul> </li> Conclusion and Next Steps</a></li> </ul> Learning a new programming language is like building muscle - sporadic gym visits won't get you far. After years of starting and abandoning Rust, I finally found a learning approach that sticks: hands-on projects that force you to write real code. This post traces my journey from tutorial hell to actually building things, and shares what worked (and what didn't) in hopes of helping others avoid the same pitfalls.</p> Why?</h2> I always wanted to learn Rust, with several false starts along the way. The appeal is clear - Rust consistently ranks as the most loved programming language in Stack Overflow's annual developer survey for the past 8 years. It promises memory safety without garbage collection, fearless concurrency, and zero-cost abstractions.</p> Modern programming languages require robust tools, and Rust delivers with its strict compiler, excellent tooling, and growing ecosystem. As a Python developer, Rust offers me a way to write performant, safe code without sacrificing productivity.</p> Previous Learning Attempts</h2> My Rust journey has been a series of starts and stops:</p> Pre-1.0 (2014): A brief experiment that ended quickly after writing a few basic functions. The borrow checker won that round.</p> 2020: My wife gifted me "The Rust Programming Language" book after overhearing my interest in Rust podcasts (particularly Rustacean Station). Finished the book and completed various small contained code examples but it was never put in to practice and started to fade.</p> 2023: Two slightly more serious attempts:</p> Explored Rust + Kafka integration for a potential work proposal</li> Started on Zero to Production in Rust</a> in October, though as I followed along with the book I was also re-implementing it in Axum</a> instead of Actix Web which the book uses. This was an improvement as the topic of the book aligned nicely with my day job as a Backend focussed engineer but I found I was learning more about Axum than making strides with my Rust fundamentals.</li> </ul> Late 2023: Returned to coding with Advent of Code. While this got me writing Rust daily for a few weeks, I ended up spending more time on AoC problem-solving patterns than Rust idioms.</p> 2024: Started preparing for another AoC attempt starting with some prep in November, more focused this time but still searching for a better learning approach.</p> Project based learning</h2> It was around this time when trying to form the habit of writing Rust with the aim of working through AoC2024 that I came across the suggestion of PNGme</a> in a response to a similar request for guidance from someone learning Rust in r/rust</a></p> PNGme</h3> PNGme bills itself as "An Intermediate Rust Project". It comprises a series of chapters each with a clear goal to build some functionality that will eventually evolve into a CLI tool for reading PNG files and then embedding or reading secret messages stored within. Almost as important as the guidance are the suite of tests to verify each chapter as you go.</p> My work</a> on this has expanded as the simple library has become a test bed to experiment with other parts of the Rust ecosystem such as:</p> Python to Rust binding with pyO3</a> by building a PNGme python library</a> from the Rust library.</li> Experimenting with GUI toolkit eframe</a> using egui</a> by building a GUI interface for PNGme</a>.</li> A html frontend for PNGme</a> using Maud and Axum.</li> Splitting a codebase into various crates in a single workspace.</li> </ul> Programming Projects for Advanced Beginners</h3> Upon finishing up PNGme the author gives some suggestions of other projects based learning resources. One was Programming Projects for Advanced Beginners</a>, a blog series by Robert Heaton which consist of prompts for small self-contained projects and some guidance on how you might approach and structure solving them.</p> This led nicely into experimenting with ratatui</a> a library designed to help you to build Text User Interfaces (TUIs) resulting in:</p> gridlife</a>: A library and TUI CLI for simulating Conway's Game of Life automatons. This was a classic case of "the interest is smaller than you think". A ratatui maintainer, Orhun Parmaksız, requested changes to my repo for use in a project of theirs. The reason I had used ratatui in the first place was due to watching a talk Orhun gave at EuroRust</a>. This also gave me the nudge I needed to publish the library to crates.io</a>.</li> A unfinished snake</a> game.</li> </ul> CodeCrafters</h3> After PNGme's success, I discovered CodeCrafters</a> through Jon Gjengset's videos. The platform offers hands-on projects where you build clones of real-world tools: Git, Redis, Docker, and more. Each project breaks down into small, testable steps with clear feedback.</p> What sets CodeCrafters apart is its focus on real-world implementations rather than toy problems. Building a BitTorrent client or Redis server forces you to understand both Rust and the underlying protocols. The automated tests provide immediate feedback, while the step-by-step progression keeps you motivated.</p> The projects I worked on via CodeCrafters are:</p> loxide</a> - An implementation of an interpreter for the Lox language by Robert Nystrom from his excellent (and free) book Crafting Interpreter</a>. This has given me a new appreciation for my 4 years studying Computer Science, with some regret that my early roles as a software engineer didn't force me to make better use of what I learned before it started to atrophy.</li> rsh</a> - A POSIX shell implementation that gave me a small peek behind the curtain to the complexity within the humble shell.</li> </ul> The real value of CodeCrafters is its focus on production-grade tools rather than toy problems. Building an interpreter forces you to understand Rust, lexing, parsing, and evaluation. Automated tests provide instant feedback, maintaining momentum.</p> Conclusion and Next Steps</h2> Looking back on the last 3-4 months I am frankly shocked by how productive I have been, in that span of time I have done more development work for my own pleasure than I had in the nearly 10 years preceding it.</p> Clear end goals keep you motivated</li> Real-world projects force you to write idiomatic code</li> Test suites provide immediate feedback</li> Building actual tools is more engaging than solving puzzles</li> </ol> Next steps:</p> Complete the Lox Interpreter from CodeCrafters, and hopefully continue onto just following the book. As the CodeCrafters Interpreter project is not complete and stops after implementing functions.</li> Contribute to some OSS project in the Rust ecosystem.</li> Build a non-trivial web service in Axum.</li> </ul> TIL: Cleaning exif data from images Unknown — Thu, 24 Oct 2024 00:00:00 +0000 When uploading images to blog from phone need to remove GPS and other private metadata. For the blog I use ExifTool</a> running exiftool -gps:all= static/*.jpg</code></p> Stepping Out of My Comfort Zone: A Backend Engineer's Journey into Blogging Unknown — Sun, 13 Oct 2024 00:00:00 +0000 Why start a blog?</a></li> The nuts and bolts</a> Why Zola?</a></li> Why Github Pages?</a></li> </ul> </li> Conclusion and Next Steps</a></li> Footnotes</a></li> </ul> As a backend engineer with over a decade of experience, I've spent countless hours diving deep into code, architecting systems, and solving complex problems. But recently, I've felt a growing urge to step out of my comfort zone and embark on a new challenge: starting a blog. In this post, I'll discuss my motivations for starting a blog and briefly explain my choice of tools: Zola and GitHub Pages.</p> Why start a blog?</h2> Preserving Knowledge across Roles</strong>: In my current and past roles I have tried to follow parts of the recommendations in Always be quitting</a> when it comes to documenting various aspects of my role. The problem with this is that even when it's a generic explainer or How To this information is lost to me once I move roles.</p> </li> Personal knowledge management</strong>: Memory is imperfect, and trying to find the same piece of information you've already encountered can be frustrating, even in this brave new world of LLMs. A blog can serve as a personal wiki or Zettelkasten system, helping me organize and retrieve information more efficiently.</p> </li> Rubber duck debugging</a></strong>1</a></sup>: This technique for solving problems when combined with posting the updates in Slack or Teams naturally leads to a set of useful textual information, transforming this into a blog item that might help someone else or maybe more importantly my future self facing the same problem again is an attactive proposition.</p> </li> Complex Systems Podcast</a></strong>: I have recently been enjoying the podcast Complex Systems hosted by Patrick McKenzie (Patio11) which in many of its episodes re-enforces the role of regular written communication and the benefits of it in critical thinking and self-examination.</p> </li> "Show your workings"</strong>: During secondary school, it was drilled into me to always show my workings in mathematics and science. The right answer wasn't always enough; demonstrating how you came to that answer was almost as important2</a></sup>. While I didn't fully appreciate this at the time, I've found throughout my professional life that following this mantra has solved or prevented many problems from developing.</p> Blogging serves as an excellent platform for "showing my workings" in my professional context. By writing about my problem-solving processes, architectural decisions, or even my learning journey, I'm not just sharing the final solution but the entire thought process behind it. This approach offers several benefits:</p> It helps me clarify my own thinking and often leads to new insights.</li> It provides a valuable resource for others who might be facing similar challenges.</li> It creates a record of my decision-making process, which can be incredibly useful when revisiting projects or defending choices later on.</li> It encourages a culture of transparency and knowledge-sharing within the tech community.</li> </ul> By treating my blog as a place to "show my workings," I'm not just creating content – I'm cultivating a habit of thorough analysis and clear communication that will serve me well throughout my career.</p> </li> </ul> The nuts and bolts</h2> Why Zola?</h3> Zola</a> is written in Rust ⚡ which aligns well with my ongoing side project to improve my Rust proficiency. I'm actively seeking opportunities to apply and deepen my Rust knowledge, so Zola's Rust foundation was a significant factor in my decision. By using Zola regularly for blogging, I hope to familiarize myself with its codebase. This familiarity could potentially lead to contributing to Zola's open-source project in the future, further enhancing my Rust skills.</li> It's a SSG (Static Site Generator) which works nicely with Github Pages.</li> Builds off strong pre-existing tooling such as Hugo, Pelican and Jeykyll.</li> It uses Tera as its template engine, which has a syntax similar to Jinja2 and Django templates. This familiarity is advantageous, as I have prior experience with both Jinja2 and Django, reducing the learning curve for creating and managing blog templates.</li> Content written in CommonMark (expanded with some Github flavour via pulldown-cmark</code></a>).</li> </ul> Why Github Pages?</h3> No real thought went into this choice, it was the default. Maybe in the future I will actually examine other options. This is mainly that it's free, easy and fits easily into my usual development flows.</p> Conclusion and Next Steps</h2> Starting this blog represents a practical step in documenting my work and sharing knowledge. By using Zola and GitHub Pages, I'm balancing the desire to learn new tools with the need for a straightforward, low-maintenance platform.</p> Moving forward, I plan to focus on:</p> Documenting solutions to specific backend engineering problems I encounter</li> Sharing brief explanations of useful techniques or patterns from my daily work</li> Posting occasional updates on my progress with learning Rust and any Zola-related insights</li> Writing about system design decisions and trade-offs from recent projects</li> </ol> I don't have a fixed posting schedule, but I aim to write when I have something concrete and useful to share. The primary goal is to create a resource that's valuable for my future self, with the added benefit of potentially helping others facing similar challenges.</p> This blog is a work in progress, and its direction may evolve based on what I find most useful and manageable alongside my primary work. If you happen to find something helpful here, all the better.</p> Footnotes</h2> ^{1</sup> Here is my current partner in debugging:</p> </div> </center> ^{2</sup> especially for the cases where you were actually wrong</p> </div>}}

How Well Do New Python Type Checkers Conform? A Deep Dive into Ty, Pyrefly, and Zuban

Unknown — Fri, 29 Aug 2025 00:00:00 +0000

The Incumbents</a></li>

ty</code> from Astral</a></li>
pyrefly</code> from Meta</a></li>
zuban</code> from David Halter</a></li> </ul> </li> </ul> </li>

Typing Conformance Suite Analysis</a>

Summary</a></li>
Review of progress</a></li>
Relevance?</a>

The Gap Between Conformance and Real-World Usage</a></li>
Practical Experience vs. Test Scores</a></li>
What This Means for Adoption Decisions</a></li>
</ul>
</li>
</ul>
</li>
Other resources to learn more</a></li>
Footnotes</a></li>
</ul>
Introduction</h1>
The Python type checking landscape is experiencing a particularly active phase of innovation in 2025. This year has witnessed the emergence of not one, not two, but three</strong> new Python type checking tools, each backed by significant pedigree and resources. While these tools have slightly different goals and philosophies, they share a common foundation: all are built in Rust with performance as a core design principle.</p>
This continued shift toward Rust-based implementations represents a significant evolution in the Python tooling ecosystem, promising faster type checking, better IDE integration, and improved developer experience for large codebases.</p>
The Incumbents</h2>
Before examining these new Rust-based tools, it's worth understanding the current landscape of Python type checkers that have established the foundation for static typing in Python:</p>
mypy</strong> - The original and most widely adopted Python type checker, developed by Jukka Lehtosalo and now maintained by the mypy team. As the reference implementation for Python's type system, mypy has shaped many of the conventions and behaviors that newer tools aim to be compatible with. It's written in Python and offers comprehensive type checking capabilities, though performance can become a bottleneck on large codebases.</p>
pyright/Pylance</strong> - Microsoft's type checker written in TypeScript/Node.js. Pyright powers the Pylance extension in VS Code and is known for its fast performance and strong IDE integration. It often implements new typing features before other checkers and provides rich editor feedback, making it popular among developers who prioritize IDE experience.</p>
pyre</strong> - A type checker from Meta, written in a mix of OCaml and Python. Designed to handle Meta's massive Python codebase, pyre introduced several performance optimizations and incremental checking capabilities. However, Meta is now developing pyrefly as pyre's successor.</p>
pytype</strong> - Google's type checker that takes a unique approach by performing type inference on unannotated Python code. Unlike other checkers that primarily validate existing type annotations, pytype can infer types from runtime behaviour and generate stub files for gradual typing adoption. Written in Python, it's particularly useful for analysing legacy codebases without type hints. However, Google announced in 2025 that pytype is being deprecated and Python 3.12 will be the last supported version of Python.</p>
The Newcomers</h2>
ty</code> from Astral</h3>
Repository:</strong> https://github.com/astral-sh/ty/</a>

Development Repository:</strong> https://github.com/astral-sh/ruff/</a></p>
Key Highlights:</strong></p>

Strong focus on gradual guarantee principles</li>
Planned tight integration with Astral's existing linting tool ruff</code>, with the aim to support type based linting rules.1</a></sup></li>
Backing from a team combining Python core developers and very experienced Rust tooling developers</li>
Incremental computation at the function level leveraging the salsa</a> library used by rust-analyzer</a>. This could be particularly important on the LSP that ty</code> provides to give very quick/responsive feedback loops in code editors.</li>
</ul>
Philosophy:</strong> Astral's approach emphasizes reliability and gradual adoption, making it easier for teams to incrementally add type checking to existing codebases without overwhelming developers with potential false positives.</p>
pyrefly</code> from Meta</h3>
Repository:</strong> https://github.com/facebook/pyrefly</a></p>
Key Highlights:</strong></p>

Successor to Meta's existing pyre</a> type checker, designed to eventually replace it</li>
Enhanced type inference capabilities by default</li>
Potentially higher upfront adoption cost due to aggressive inference which might flag issues with correct code</li>
Backed by Meta's substantial engineering resources and real-world usage at scale</li>
</ul>
Philosophy:</strong> Meta prioritizes powerful inference and catching more potential issues out of the box, even if this means a steeper learning curve for teams new to type checking.</p>
zuban</code> from David Halter</h3>
Homepage:</strong> https://zubanls.com/</a>

Documentation:</strong> https://docs.zubanls.com/en/latest/</a></p>
Key Highlights:</strong></p>

Created by the author of the popular Python LSP tool jedi</code></li>
Aims for high-degree of compatibility with mypy</code> to make adoption in large existing codebases seamless.</li>
Not FOSS2</a></sup>, will require a license for codebases above 1.5 MB (~50,000 lines of code)3</a></sup></del>Update (September 2025):</strong> Now open source under the AGPL license, though commercial licensing is available for business who prefer to avoid AGPL compliance.</li>
Currently maintained by a single author seems a potential risk to long-term sustainability as Python typing does not stand still.</li>
</ul>
Philosophy:</strong> Zuban aims to provide the smoothest possible migration path from existing type checkers, particularly mypy</code>, making it attractive for organizations with substantial existing typed codebases.</p>
Typing Conformance Suite Analysis</h1>
The Python Typing Council maintains a Conformance test suite</a> which validates the behaviour of static type checkers against the expectations defined in the Python typing specification</a>.</p>
ty</code> and pyrefly</code> have not yet been added to the conformance suite, so it's harder to establish a baseline for their progress on this front.</p>
To help resolve this gap I have expanded the current test harness to support both of them</a>4</a></sup>, this also adds some additional debugging information to the html report to show the split between false negatives and false positives that the suite has detected in each test case.</p>
I've also included a local build of ty</code> for two main reasons:</p>

I've wanted to investigate contributing to ty</code> and/or ruff</code> for a while, so this was good impetus to get things set up.</li>
ty</code> releases are cut relatively infrequently and I am impatient.</li>
</ul>
Summary</h2>
Generated 29/08/2025</strong></p>

NOTE</p>
The following section is slightly unfair, all of these tools are in alpha and only one of these tools (Zuban) has opted into the Conformance suite. This was mainly driven by own curiosity and seemed like an interesting project to understand the conformance test suite better.</p>
That being said even though ty</code> is lagging on this metric at the moment it is still the type checker that I am most excited to use long-term because of the quality of the tooling Astral has built so far.</p>
</blockquote>
Type Checker</th> Total Test Case Passes</th> Total Test Case Partial</th> Total False Positives</th> Total False Negatives</th></tr></thead>

zuban 0.0.20</td> 97</td> 42</td> 152</td> 89</td></tr>
ty 0.0.1-alpha.19 (e9cb838b3 2025-08-19)</td> 20</td> 119</td> 371</td> 603</td></tr>
Local:ty ruff/0.12.11+27 (0bf5d2a20 2025-08-29)</td> 20</td> 119</td> 370</td> 590</td></tr>
pyrefly 0.30.0</td> 81</td> 58</td> 100</td> 187</td></tr>
</tbody></table>
Review of progress</h2>
zuban</code> has a lead, having full Pass</code> on ~69% of test cases, compared with ~15% for ty</code> and ~58% for pyrefly</code>. Which makes sense as though it's released in a similar time period to ty</code> and pyrefly</code> it has been in active development in private for several years.</p>
The thing that surprised me more was how much progress pyrefly</code> has made when compared to ty</code>. Both broke cover and released their first advertised alpha builds around the same time, in the run up to PyCon 2025. This can maybe be partially explained from a point raised in Edward Li's excellent blog post on the Typing Summit at PyCon 2025</a> which mentions that the pyrefly</code> team devoted a lot of up-front time to solving some of the hard problems, such as generics.</p>
Relevance?</h2>
I initially came across the conformance test suite because ty</code> runs every PR against the test suite and diffs the PR results against the results from main</code> to ensure changes are desired. From this it's become a surprisingly useful learning tool for some of the more advanced typing topics, but the advanced nature of these topics raises an important question: how relevant is the conformance suite pass rate for the average Python developer?</strong></p>
The Gap Between Conformance and Real-World Usage</h3>
The conformance test suite focuses heavily on advanced typing features that, while important for the specification, may not reflect the day-to-day typing needs of most Python codebases. Many of the test cases cover complex scenarios involving:</p>

Advanced generic variance and bounds</li>
Complex protocol inheritance hierarchies</li>
Edge cases in structural subtyping</li>
Intricate interactions between multiple typing features</li>
</ul>
In contrast, the majority of Python codebases primarily use:</p>

Basic type annotations (str</code>, int</code>, List[str]</code>, etc.)</li>
Simple class hierarchies</li>
Optional types and Union types</li>
Basic generic containers</li>
</ul>
Practical Experience vs. Test Scores</h3>
My experience using the ty</code> VSCode extension in place of Pylance</code> across various projects and libraries tells a different story than the conformance test scores suggest. Despite ty</code>'s relatively low 15% full pass rate, it has been surprisingly effective at catching real bugs and providing useful feedback for common typing patterns.</p>
This suggests that while conformance test coverage is important for specification compliance and handling edge cases, it may not be the best predictor of a type checker's utility for everyday Python development. The features that matter most for typical codebases appear to be working well across all three tools, even when they struggle with more esoteric typing scenarios.</p>
What This Means for Adoption Decisions</h3>
For teams evaluating these type checkers, the conformance scores provide valuable insight into specification compliance, but shouldn't be the sole deciding factor. Consider:</p>

For greenfield projects</strong>: Any of these tools will likely handle your immediate needs well</li>
For large, complex codebases or libraries leaning on more esoteric generic patterns</strong>: Higher conformance scores may indicate better handling of advanced patterns you might encounter</li>
For teams new to typing</strong>: The difference in conformance scores may be less relevant than IDE integration, error message quality, and performance</li>
</ul>
Other resources to learn more</h1>

Edward Li's excellent blog post comparing ty</code> and pyrefly</code></a> which also contains the videos recorded at the PyCon typing summit which both ty</code> and pyrefly</code> gave presentations at.</li>
Happy Path Programming - 114 ty: Fast Python Type Checking with Carl Meyer</a></li>
Happy Path Programming - 115 More Python Type Checking! Pyrefly with Aaron Pollack & Steven Troxler</a></li>
Talk Python - ty: Astral's New Type Checker (Formerly Red-Knot)</a></li>
</ul>
Footnotes</h1>
^1</sup>Which can be demonstrated in the open issues</a> on ruff tagged with type-inference</code> which are bugs or new features that can only be resolved with ruff</code> having access to deeper type inference data that ty</code> can supply.</p>
</div>
^2</sup>David has indicated a plan to make source available in the future</a> when adding Zuban to the Python typing conformance suite.</p>
</div>
^{3</sup>
Full pricing information at: https://zubanls.com/pricing/</del> Update (September 2025):</strong> Pricing is now available on request for the non-AGPL license.</p>
</div>}^{4</sup>
This is just for this blog post, no plans to seek merging this.</p>
</div>}
TIL: Flowistry tool to understand information flow in Rust
Unknown — Thu, 28 Aug 2025 00:00:00 +0000
When watching the excellent talk Rust for Everyone</a> by Will Crichton the item that stood out as potentially most useful day to day was Flowistry</a>. It is a VSCode extension, when a user clicks a variable the extension greys out all code that the variable does not interact with, it seems like it could be very useful when debugging.</p>


About
Unknown — Wed, 27 Aug 2025 00:00:00 +0000
I am Rob Hand, a Staff Software Engineer currently working on Personalisation at M&S. Enjoy learning new things and using those things to solve more complex problems.</p>
For an (hopefully) more up to date snapshot see Now</a> page.</p>
You can find me on GitHub</a> or send me a message on Bluesky</a>. My CV is available here</a>.</p>


Boardgames
Unknown — Wed, 27 Aug 2025 00:00:00 +0000



Now
Unknown — Wed, 27 Aug 2025 00:00:00 +0000
(This is a now page</a>, and if you have your own site, you should make one</a>, too.)</strong></p>

reading: Children of Memory - Adrian Tchaikovsky</li>
watching: Nothing regular since Andor Season 2, looking forward to season 2 of Fallout</li>
learning: Rust🦀</a> and Gleam⭐</a>.</li>
writing: Attempting to build the habit of Journaling with the help of LogSeq</a></li>
listening: See on Last.fm</a></li>
playing: Mainly Roguelike games that require less time commitment such as: Astral Ascent</a> or Risk of Rain 2</a></li>
</ul>
Continuing to try and spend more time on hobbies instead of doomsrolling or watching too much YouTube once the work day is over and the kids are put to bed. Not always</del> Rarely</strong> succeeding.</p>
Current Projects:</p>

loxide</a> - Implementing an interpreter for lox language from https://craftinginterpreters.com/ in Rust. At the point of adding support for user defined functions.</li>
z2p-axum</a> - Following along with the Zero to Prod book but re-implementing the application in Axum instead of Actix.</li>
</ul>
Finished:</p>

chip8</a> - A Chip-8</a> interpreter written in no-std Rust.</li>
gridlife</a> - A library for Conways Game of Life in Rust with a TUI to simulate random configurations.</li>
pngme</a> -  Implementation of https://jrdngr.github.io/pngme_book/ which became my go to testing/learning project. Expanded the original with: GUI, Python library binding and crate splitting the project.</li>
</ul>
On Hold:</p>

snake</a> - A snake clone TUI built using Ratatui</li>
</ul>
Last updated on 27th August 2025.</p>


TIL: Now page
Unknown — Wed, 27 Aug 2025 00:00:00 +0000
Added a Now</a> page to blog based on nownownow</a>. Since I don't post on social media this seemed like a nice way to share what I am up to at the moment (to my many many readers... crickets</strong></em>)</p>


Bridging Python & Rust: A Walkthrough of using Py03
Unknown — Sun, 18 May 2025 00:00:00 +0000

Bridging Python and Rust: A Practical Guide with PyO3</a></li>
Project Structure</a></li>
Step 1: Exposing Rust Functions with PyO3</a></li>
Step 2: Building and Packaging</a></li>
Step 3: Using the Library from Python</a></li>
Step 4: Handling Errors</a></li>
Wrapping Up</a></li>
</ul>
Bridging Python and Rust: A Practical Guide with PyO3</h1>
Sometimes Python just isn't fast enough, or you want to reuse some Rust code without rewriting it. PyO3</a> makes it surprisingly easy to call Rust from Python (or less commonly vice-versa). Here’s how I created pngme-python</a> crate, to expose my already existing pngme</a> Rust crate as a python library.</p>
Project Structure</h2>
pngme/
</span>├── src/
</span>│   └── lib.rs
</span>├── Cargo.toml
</span>pngme-python/
</span>├── src/
</span>│   └── lib.rs
</span>├── tests/
</span>│   └── test_pngme.py
</span>├── Cargo.toml
</span>├── pyproject.toml
</span>├── README.md
</span>
</span></code></pre>

pngme/src/lib.rs</strong>: The original Rust code with PNG manipulation functionality, that we want to expose as a Python library</li>
pngme-python/src/lib.rs</strong>: The Py03 bindings.</li>
test_pngme.py</strong>: Python tests to verify the bindings work correctly</li>
pngme-python/src/Cargo.toml</strong>: The build configuration for the Rust portion of the Py03/Maturin build process.</li>
pyproject.toml</strong>: Python packaging configuration for Maturin</li>
</ul>
Step 1: Exposing Rust Functions with PyO3</h2>
PyO3 lets you turn Rust functions into Python-callable methods with minimal fuss. Here’s a trimmed-down version of the encode</code>, decode</code>, and remove</code> functions from src/lib.rs</a>:</p>

</span>use </span>pyo3::prelude::</span>*</span>;
</span>
</span>#[pymodule]
</span>#[pyo3(name </span>= </span>"pngme"</span>)]
</span>mod </span>pngme_python {
</span>    </span>use </span>pyo3::exceptions::{PyFileNotFoundError, PyIOError, PyValueError};
</span>    </span>use </span>pyo3::{prelude::</span>*</span>, PyResult};
</span>    </span>use </span>std::path::PathBuf;
</span>
</span>    </span>use </span>pngme_lib::{decode </span>as</span> png_decode, encode </span>as</span> png_encode, remove </span>as</span> png_remove, Error};
</span>
</span>    #[pyfunction]
</span>    </span>pub fn </span>encode</span>(</span>path</span>: PathBuf, </span>chunk_type</span>: String, </span>message</span>: String) -> PyResult<()> {
</span>        </span>let</span> result </span>= </span>png_encode</span>(</span>&</span>path, </span>&</span>chunk_type, message);
</span>        </span>match</span> result {
</span>            </span>Ok</span>(</span>_</span>) </span>=> </span>Ok</span>(()),
</span>            </span>Err</span>(e) </span>=> </span>match</span> e {
</span>                Error::FileNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyFileNotFoundError::new_err(e.</span>to_string</span>())),
</span>                Error::Read { source: s } </span>=> </span>Err</span>(PyIOError::new_err(s.</span>to_string</span>())),
</span>                Error::PNGParse </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>                Error::InvalidChunkType { source: s, </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(s.</span>to_string</span>())),
</span>                Error::PNGWrite { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>                Error::ChunkNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>                Error::StrConversion </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>            },
</span>        }
</span>    }
</span>
</span>    #[pyfunction]
</span>    </span>pub fn </span>decode</span>(</span>path</span>: PathBuf, </span>chunk_type</span>: String) -> PyResult<</span>String</span>> {
</span>        </span>let</span> result </span>= </span>png_decode</span>(</span>&</span>path, </span>&</span>chunk_type);
</span>        </span>match</span> result {
</span>            </span>Ok</span>(msg) </span>=> </span>Ok</span>(msg),
</span>            </span>Err</span>(e) </span>=> </span>match</span> e {
</span>                Error::FileNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyFileNotFoundError::new_err(e.</span>to_string</span>())),
</span>                Error::Read { </span>.. </span>} </span>=> </span>Err</span>(PyIOError::new_err(e.</span>to_string</span>())),
</span>                Error::PNGParse </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>                Error::InvalidChunkType { source: s, </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(s.</span>to_string</span>())),
</span>                Error::PNGWrite { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>                Error::ChunkNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>                Error::StrConversion </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>            },
</span>        }
</span>    }
</span>
</span>    #[pyfunction]
</span>    </span>pub fn </span>remove</span>(</span>path</span>: PathBuf, </span>chunk_type</span>: String) -> PyResult<()> {
</span>        </span>let</span> result </span>= </span>png_remove</span>(</span>&</span>path, </span>&</span>chunk_type);
</span>        </span>match</span> result {
</span>            </span>Ok</span>(</span>_</span>) </span>=> </span>Ok</span>(()),
</span>            </span>Err</span>(e) </span>=> </span>match</span> e {
</span>                Error::FileNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyFileNotFoundError::new_err(e.</span>to_string</span>())),
</span>                Error::Read { </span>.. </span>} </span>=> </span>Err</span>(PyIOError::new_err(e.</span>to_string</span>())),
</span>                Error::PNGParse </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>                Error::InvalidChunkType { source: s, </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(s.</span>to_string</span>())),
</span>                Error::PNGWrite { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>                Error::ChunkNotFound { </span>.. </span>} </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>                Error::StrConversion </span>=> </span>Err</span>(PyValueError::new_err(e.</span>to_string</span>())),
</span>            },
</span>        }
</span>    }
</span>}
</span></code></pre>
The key parts of this implementation:</p>

The #[pymodule]</code> macro creates a Python module from the Rust module it encloses.</li>
Each #[pyfunction]</code> within the Rust module adds a Rust function to Python the #[pymodule]</code>.</li>
Each Rust error type must be handled and mapped to the appropriate Python exceptions.</li>
</ul>
Step 2: Building and Packaging</h2>
Maturin</a> handles compiling the Rust code and packaging it as a Python wheel. Two configuration files control this process:</p>
Cargo.toml</strong></p>
[package]
</span>name </span>= </span>"pngme-python"
</span>version </span>= </span>"0.1.0"
</span>edition </span>= </span>"2021"
</span>
</span>[lib]
</span>name </span>= </span>"pngme"
</span>crate-type </span>= [</span>"cdylib"</span>]
</span>
</span>[dependencies]
</span>pyo3 </span>= { </span>version </span>= </span>"0.18.3"</span>, </span>features </span>= [</span>"extension-module"</span>] }
</span>pngme-lib </span>= { </span>path </span>= </span>"../pngme" </span>}
</span></code></pre>
pyproject.toml</strong></p>
[build-system]
</span>requires </span>= [</span>"maturin>=1.0,<2.0"</span>]
</span>build-backend </span>= </span>"maturin"
</span>
</span>[project]
</span>name </span>= </span>"pngme"
</span>version </span>= </span>"0.1.0"
</span>description </span>= </span>"Python bindings for pngme"
</span>readme </span>= </span>"README.md"
</span>
</span>[tool.maturin]
</span>features </span>= [</span>"pyo3/extension-module"</span>]
</span></code></pre>
Building is as simple as:</p>

</span>maturin develop
</span></code></pre>
This creates a Python wheel that you can use directly or publish to PyPI.</p>
Step 3: Using the Library from Python</h2>
Once built, and installed, just import and use the module in Python. An example can be found in tests/test_pngme.py</a>:</p>

</span>import </span>pngme
</span>
</span>def test_pngme_encode</span>():
</span>    file_location </span>= </span>"./crates/pngme-python/tests/dice.png"
</span>    pngme.encode(file_location, </span>"ruSt"</span>, </span>"some message"</span>)
</span>    output </span>= </span>pngme.decode(file_location, </span>"ruSt"</span>)
</span>    </span>assert </span>output </span>== </span>"some message"
</span>    pngme.remove(file_location, </span>"ruSt"</span>)
</span>    nothing </span>= </span>pngme.decode(file_location, </span>"ruSt"</span>)
</span>    </span>assert </span>nothing </span>== </span>"No secret message found"
</span></code></pre>
Step 4: Handling Errors</h2>
PyO3 lets you map Rust errors to Python exceptions, so Python users get idiomatic error messages as shown below:</p>

</span>import </span>pytest
</span>
</span>def test_pngme_unknown_file</span>():
</span>    </span>with </span>pytest.raises(</span>FileNotFoundError</span>) </span>as </span>exc:
</span>        pngme.encode(</span>"unknown.png"</span>, </span>"ruSt"</span>, </span>"some message"</span>)
</span>    </span>assert </span>'File not found "unknown.png"' </span>in </span>str</span>(exc.value)
</span></code></pre>
Wrapping Up</h2>
PyO3 makes it easy to bring Rust’s speed and safety to Python, with natural error handling and a smooth workflow. If you want to squeeze more performance out of Python or reuse Rust code, give it a try.</p>

References:</strong></p>

PyO3 Documentation</a></li>
Maturin Documentation</a></li>
pngme-python Source Code</a></li>
</ul>


Project-Based Learning: The Method That Made Rust Finally Click
Unknown — Fri, 07 Feb 2025 00:00:00 +0000

Why?</a></li>
Previous Learning Attempts</a></li>
Project based learning</a>

PNGme</a></li>
CodeCrafters</a></li>
</ul>
</li>
Conclusion and Next Steps</a></li>
</ul>
Learning a new programming language is like building muscle - sporadic gym visits won't get you far. After years of starting and abandoning Rust, I finally found a learning approach that sticks: hands-on projects that force you to write real code. This post traces my journey from tutorial hell to actually building things, and shares what worked (and what didn't) in hopes of helping others avoid the same pitfalls.</p>
Why?</h2>
I always wanted to learn Rust, with several false starts along the way. The appeal is clear - Rust consistently ranks as the most loved programming language in Stack Overflow's annual developer survey for the past 8 years. It promises memory safety without garbage collection, fearless concurrency, and zero-cost abstractions.</p>
Modern programming languages require robust tools, and Rust delivers with its strict compiler, excellent tooling, and growing ecosystem. As a Python developer, Rust offers me a way to write performant, safe code without sacrificing productivity.</p>
Previous Learning Attempts</h2>
My Rust journey has been a series of starts and stops:</p>
Pre-1.0 (2014): A brief experiment that ended quickly after writing a few basic functions. The borrow checker won that round.</p>
2020: My wife gifted me "The Rust Programming Language" book after overhearing my interest in Rust podcasts (particularly Rustacean Station). Finished the book and completed various small contained code examples but it was never put in to practice and started to fade.</p>
2023: Two slightly more serious attempts:</p>

Explored Rust + Kafka integration for a potential work proposal</li>
Started on Zero to Production in Rust</a> in October, though as I followed along with the book I was also re-implementing it in Axum</a> instead of Actix Web which the book uses. This was an improvement as the topic of the book aligned nicely with my day job as a Backend focussed engineer but I found I was learning more about Axum than making strides with my Rust fundamentals.</li>
</ul>
Late 2023: Returned to coding with Advent of Code. While this got me writing Rust daily for a few weeks, I ended up spending more time on AoC problem-solving patterns than Rust idioms.</p>
2024: Started preparing for another AoC attempt starting with some prep in November, more focused this time but still searching for a better learning approach.</p>
Project based learning</h2>
It was around this time when trying to form the habit of writing Rust with the aim of working through AoC2024 that I came across the suggestion of PNGme</a> in a response to a similar request for guidance from someone learning Rust in r/rust</a></p>
PNGme</h3>
PNGme bills itself as "An Intermediate Rust Project". It comprises a series of chapters each with a clear goal to build some functionality that will eventually evolve into a CLI tool for reading PNG files and then embedding or reading secret messages stored within. Almost as important as the guidance are the suite of tests to verify each chapter as you go.</p>
My work</a> on this has expanded as the simple library has become a test bed to experiment with other parts of the Rust ecosystem such as:</p>

Python to Rust binding with pyO3</a> by building a PNGme python library</a> from the Rust library.</li>
Experimenting with GUI toolkit eframe</a> using egui</a> by building a GUI interface for PNGme</a>.</li>
A html frontend for PNGme</a> using Maud and Axum.</li>
Splitting a codebase into various crates in a single workspace.</li>
</ul>
Programming Projects for Advanced Beginners</h3>
Upon finishing up PNGme the author gives some suggestions of other projects based learning resources. One was Programming Projects for Advanced Beginners</a>, a blog series by Robert Heaton which consist of prompts for small self-contained projects and some guidance on how you might approach and structure solving them.</p>
This led nicely into experimenting with ratatui</a> a library designed to help you to build Text User Interfaces (TUIs) resulting in:</p>

gridlife</a>: A library and TUI CLI for simulating Conway's Game of Life automatons. This was a classic case of "the interest is smaller than you think". A ratatui maintainer, Orhun Parmaksız, requested changes to my repo for use in a project of theirs. The reason I had used ratatui in the first place was due to watching a talk Orhun gave at EuroRust</a>. This also gave me the nudge I needed to publish the library to crates.io</a>.</li>
A unfinished snake</a> game.</li>
</ul>
CodeCrafters</h3>
After PNGme's success, I discovered CodeCrafters</a> through Jon Gjengset's videos. The platform offers hands-on projects where you build clones of real-world tools: Git, Redis, Docker, and more. Each project breaks down into small, testable steps with clear feedback.</p>
What sets CodeCrafters apart is its focus on real-world implementations rather than toy problems. Building a BitTorrent client or Redis server forces you to understand both Rust and the underlying protocols. The automated tests provide immediate feedback, while the step-by-step progression keeps you motivated.</p>
The projects I worked on via CodeCrafters are:</p>

loxide</a> - An implementation of an interpreter for the Lox language by Robert Nystrom from his excellent (and free) book Crafting Interpreter</a>. This has given me a new appreciation for my 4 years studying Computer Science, with some regret that my early roles as a software engineer didn't force me to make better use of what I learned before it started to atrophy.</li>
rsh</a> - A POSIX shell implementation that gave me a small peek behind the curtain to the complexity within the humble shell.</li>
</ul>
The real value of CodeCrafters is its focus on production-grade tools rather than toy problems. Building an interpreter forces you to understand Rust, lexing, parsing, and evaluation. Automated tests provide instant feedback, maintaining momentum.</p>
Conclusion and Next Steps</h2>
Looking back on the last 3-4 months I am frankly shocked by how productive I have been, in that span of time I have done more development work for my own pleasure than I had in the nearly 10 years preceding it.</p>

Clear end goals keep you motivated</li>
Real-world projects force you to write idiomatic code</li>
Test suites provide immediate feedback</li>
Building actual tools is more engaging than solving puzzles</li>
</ol>
Next steps:</p>

Complete the Lox Interpreter from CodeCrafters, and hopefully continue onto just following the book. As the CodeCrafters Interpreter project is not complete and stops after implementing functions.</li>
Contribute to some OSS project in the Rust ecosystem.</li>
Build a non-trivial web service in Axum.</li>
</ul>



TIL: Cleaning exif data from images
Unknown — Thu, 24 Oct 2024 00:00:00 +0000
When uploading images to blog from phone need to remove GPS and other private metadata. For the blog I use ExifTool</a> running exiftool -gps:all= static/*.jpg</code></p>


Stepping Out of My Comfort Zone: A Backend Engineer's Journey into Blogging
Unknown — Sun, 13 Oct 2024 00:00:00 +0000

Why start a blog?</a></li>
The nuts and bolts</a>

Why Zola?</a></li>
Why Github Pages?</a></li>
</ul>
</li>
Conclusion and Next Steps</a></li>
Footnotes</a></li>
</ul>
As a backend engineer with over a decade of experience,
I've spent countless hours diving deep into code, architecting systems, and solving complex problems.
But recently, I've felt a growing urge to step out of my comfort zone and embark on a new challenge: starting a blog.
In this post, I'll discuss my motivations for starting a blog and briefly explain my choice of tools: Zola and GitHub Pages.</p>
Why start a blog?</h2>


Preserving Knowledge across Roles</strong>: In my current and past roles I have tried to follow parts of the recommendations in
Always be quitting</a> when it comes to documenting various aspects of my role.
The problem with this is that even when it's a generic explainer or How To this information is lost to me once I move roles.</p>
</li>

Personal knowledge management</strong>: Memory is imperfect, and trying to find the same piece of information
you've already encountered can be frustrating, even in this brave new world of LLMs.
A blog can serve as a personal wiki or Zettelkasten system,
helping me organize and retrieve information more efficiently.</p>
</li>

Rubber duck debugging</a></strong>1</a></sup>: This technique for solving problems when
combined with posting the updates in Slack or Teams naturally leads to a set of useful
textual information, transforming this into a blog item that might help someone else or maybe more importantly my future self
facing the same problem again is an attactive proposition.</p>
</li>

Complex Systems Podcast</a></strong>: I have recently been enjoying the podcast Complex Systems hosted by Patrick McKenzie (Patio11)
which in many of its episodes re-enforces the role of regular written communication and the benefits of it in
critical thinking and self-examination.</p>
</li>

"Show your workings"</strong>: During secondary school, it was drilled into me to always show my workings in mathematics and science.
The right answer wasn't always enough; demonstrating how you came to that answer was almost as important2</a></sup>. While I didn't fully appreciate this at the time, I've found throughout my professional life that following this mantra has solved or prevented many problems from developing.</p>
Blogging serves as an excellent platform for "showing my workings" in my professional context.
By writing about my problem-solving processes, architectural decisions, or even my learning journey,
I'm not just sharing the final solution but the entire thought process behind it. This approach offers several benefits:</p>

It helps me clarify my own thinking and often leads to new insights.</li>
It provides a valuable resource for others who might be facing similar challenges.</li>
It creates a record of my decision-making process, which can be incredibly useful when revisiting projects or defending choices later on.</li>
It encourages a culture of transparency and knowledge-sharing within the tech community.</li>
</ul>
By treating my blog as a place to "show my workings," I'm not just creating content – I'm cultivating a habit of thorough analysis and clear communication that will serve me well throughout my career.</p>
</li>
</ul>
The nuts and bolts</h2>
Why Zola?</h3>

Zola</a> is written in Rust ⚡ which aligns well with my ongoing side project to improve my Rust proficiency. I'm actively seeking opportunities to apply and deepen my Rust knowledge, so Zola's Rust foundation was a significant factor in my decision. By using Zola regularly for blogging, I hope to familiarize myself with its codebase. This familiarity could potentially lead to contributing to Zola's open-source project in the future, further enhancing my Rust skills.</li>
It's a SSG (Static Site Generator) which works nicely with Github Pages.</li>
Builds off strong pre-existing tooling such as Hugo, Pelican and Jeykyll.</li>
It uses Tera as its template engine, which has a syntax similar to Jinja2 and Django templates. This familiarity is advantageous, as I have prior experience with both Jinja2 and Django, reducing the learning curve for creating and managing blog templates.</li>
Content written in CommonMark (expanded with some Github flavour via pulldown-cmark</code></a>).</li>
</ul>
Why Github Pages?</h3>
No real thought went into this choice, it was the default. Maybe in the future I will actually
examine other options. This is mainly that it's free, easy and fits easily into my usual
development flows.</p>
Conclusion and Next Steps</h2>
Starting this blog represents a practical step in documenting my work and sharing knowledge. By using Zola and GitHub Pages, I'm balancing the desire to learn new tools with the need for a straightforward, low-maintenance platform.</p>
Moving forward, I plan to focus on:</p>

Documenting solutions to specific backend engineering problems I encounter</li>
Sharing brief explanations of useful techniques or patterns from my daily work</li>
Posting occasional updates on my progress with learning Rust and any Zola-related insights</li>
Writing about system design decisions and trade-offs from recent projects</li>
</ol>
I don't have a fixed posting schedule, but I aim to write when I have something concrete and useful to share. The primary goal is to create a resource that's valuable for my future self, with the added benefit of potentially helping others facing similar challenges.</p>
This blog is a work in progress, and its direction may evolve based on what I find most useful and manageable alongside my primary work. If you happen to find something helpful here, all the better.</p>
Footnotes</h2>
^{1</sup>
Here is my current partner in debugging:</p>
</div>


</center>
^{2</sup>
especially for the cases where you were actually wrong</p>
</div>}}

TIL: Flowistry tool to understand information flow in Rust

About

Boardgames

Now

TIL: Now page

Bridging Python & Rust: A Walkthrough of using Py03

Project-Based Learning: The Method That Made Rust Finally Click

Project based learning</h2>
It was around this time when trying to form the habit of writing Rust with the aim of working through AoC2024 that I came across the suggestion of PNGme</a> in a response to a similar request for guidance from someone learning Rust in r/rust</a></p>

TIL: Cleaning exif data from images

Stepping Out of My Comfort Zone: A Backend Engineer's Journey into Blogging

The nuts and bolts</h2>

Why Github Pages?</h3>
No real thought went into this choice, it was the default. Maybe in the future I will actually examine other options. This is mainly that it's free, easy and fits easily into my usual development flows.</p>

Footnotes</h2>
^{1</sup>
Here is my current partner in debugging:</p>
</div>

</center>
^{2</sup>
especially for the cases where you were actually wrong</p>
</div>}}

Rob's Blog | Python • Rust • Ramblings?

Writing more, writing TILs

How Well Do New Python Type Checkers Conform? A Deep Dive into Ty, Pyrefly, and Zuban

Rob's Blog | Python • Rust • Ramblings?

Writing more, writing TILs

How Well Do New Python Type Checkers Conform? A Deep Dive into Ty, Pyrefly, and Zuban

TIL: Flowistry tool to understand information flow in Rust

About

Boardgames

Now

TIL: Now page

Bridging Python & Rust: A Walkthrough of using Py03

Step 4: Handling Errors</h2> PyO3 lets you map Rust errors to Python exceptions, so Python users get idiomatic error messages as shown below:</p>

Wrapping Up</h2> PyO3 makes it easy to bring Rust’s speed and safety to Python, with natural error handling and a smooth workflow. If you want to squeeze more performance out of Python or reuse Rust code, give it a try.</p> References:</strong></p>

Project-Based Learning: The Method That Made Rust Finally Click

TIL: Cleaning exif data from images

Stepping Out of My Comfort Zone: A Backend Engineer's Journey into Blogging

The nuts and bolts</h2>

Why Github Pages?</h3> No real thought went into this choice, it was the default. Maybe in the future I will actually examine other options. This is mainly that it's free, easy and fits easily into my usual development flows.</p>

Footnotes</h2> 1</sup> Here is my current partner in debugging:</p> </div> </center> 2</sup> especially for the cases where you were actually wrong</p> </div>

Why Github Pages?</h3>
No real thought went into this choice, it was the default. Maybe in the future I will actually examine other options. This is mainly that it's free, easy and fits easily into my usual development flows.</p>

Footnotes</h2>
^{1</sup>
Here is my current partner in debugging:</p>
</div>

</center>
^{2</sup>
especially for the cases where you were actually wrong</p>
</div>}}