TL;DR: I got mad about Sony's unreliable geotagging apps, reverse engineered the protocol, and built my own: Geotag Alpha. Get the beta here.

I picked up a Sony a7 IV earlier this year to replace my aging a7R II, and I was initially pleased about the geotagging feature as I tend to shoot most of my photos while travelling, and it'll be nice to know exactly where I took them.

However, I quickly noticed that the Sony Creators' app struggled to maintain a connection to my camera, especially after I turned the camera off (to save battery) and turning it back on. Having the app in the foreground worked, but I'm not going to keep their app open in the foreground every time I wanted to connect.

I got real mad about it, to the point where I nerd-sniped myself the day before I was leaving for a trip to Japand and Taiwan, and set out to figure out how to build a more reliable geotagging solution. With a bunch of reverse engineering and help from other people's posts who've tried to do something similar, I managed to hack together working prototype (where the UI was the default "Hello world" screen) by 11pm that evening.

I've called it Geotag Alpha (currently in Testflight), and it improves upon the Creators' app functionality by:

• Properly handling connecting to the camera while in the background
• Better energy efficiency. Location updates are only pushed when it changes or at the interval the camera needs, rather than the 7s internal Sony uses.
• Support for geotagging multiple cameras simultaneously (coming soon!)

I've tested it with my own Sony a7 IV, and have confirmed it works on Sony a7R IV, Sony a7 IIIs so far.

If you geotag your cameras and you also struggle with the official Sony Creators and Imaging Edge Mobile apps not being reliable, give Geotag Alpha a go!

Join the Testflight beta and let me know if it works for you!

TL;DR: Reflection Metadata Level set to "Off" (potentially a default from older Xcode projects) can cause @State/@Binding to have issues around getting the variable to update.

After banging my head against the wall for a couple of hours, I managed to luck out on an issue that seemed extremely fucking weird and I had much hate for SwiftUI until I figured out the problem.

I was in the middle of building a release to test update mechanism on Shortcat when I noticed the onboarding screen's Next button would not appear to do anything.

The root symptom was the step binding was somehow not being set to .second, and was stuck in .first, confirmed by print statements (trusted tool) before and after:

Button(action: {
    withAnimation {
        print("before: \(step.rawValue)")
        step = .second
        print("after: \(step.rawValue)")
    }
}) {

This resulted in:

before: first
after: first

Which itself is weird as fuck, but what's weirder is it was only happening when being built for Release. For me, if a build setting is causing code to somehow have read-only variables, something is seriously cooked.

I cleaned my build folder, updated Xcode, made a new project and just put the relevant views into a fresh project, set it to Release, and... it worked fine.

What the fuck?

I tried cutting bits of code out that wasn't in the fresh project (dependencies etc) and nothing.

I had to resort to looking at seeing what build options were different between Development and Release, and eventually happened to change the Reflection Metadata Level option to All... and it worked!?

A quick search for swift reflection metadata level, I spotted a link that said "SwiftUI state not updated in older project" and I figured that absolutely must be it.

The question was 1 year and 10 months old (at time of writing). I gave the relevant answer and upvote, and figured I'd write a post about it so hopefully someone using the keywords I used will also discover the answer one day.

This post describes how I managed to fix a friend's save where he was trying meet Hanako at Embers, but the elevator/lift that's being guarded by two bouncers is marked "Off" and he couldn't enter it. I've heard of another issue where the lift is open, but the bouncer is stopping you from entering. I'm not sure if this helps with that issue.

Please note that I am not providing support for this issue, and I am not responsible for any damage this can cause to your save game/computer/etc. Check out the CP77 Modding Tools Discord if you need help.

You need a recent CyberEngineTweaks installed and working (see other resources for how to install as that's beyond the scope of this post), as this workaround requires poking at game state in the console that CET provides. Instructions tested below on v1.06 of the game and a CET build newer than the 5th of Jan, 2021.

1. Go to the Embers lift with the two bouncers, and trigger the Point Of No Return dialog by walking up to the lift.
2. Open the CET Console with tilde (`) or whatever it is on your keyboard
3. Teleport inside the lift with: Game.TeleportPlayerToPosition(-1794.316040,-535.862915,10.11386)
4. Look at the elevator control panel
5. Run ts = Game.GetTargetingSystem(); lift = ts:GetLookAtObject(Game.GetPlayer(),false,false):GetDevicePS(); print(lift:GetDeviceState())
6. The console should display EDeviceStatus : (4294967295)
7. Look away from elevator control panel
8. Run lift:SetDeviceState(1)
9. Look at control panel, and it should update and allow you to go to the Embers floor.

Things that didn't work

• Thanks to @SirBitesalot's fact dump, I found a q115_embers_elevator_unlocked flag which I was certain that was it cause it was not set in friend's file. However, this didn't fix the issue.
• Teleporting into Embers itself: NPCs were there, but do not react. I believe there's a trigger when the elevator doors open.
• Gradually teleporting up the lift well: This does trigger a conversation with Johnny, but doesn't seem to progress further, even teleporting into Embers after that.

Thanks to the modding community for making this remotely possible. No thanks to C++ which still doesn't have a nice way to join an array of strings with a delimiter in its standard library.

This is a continuation of my deep-dive into understanding print quality issues when printing over USB with Octoprint, and adding buffer monitoring to Marlin.

Now that I had an objective mechanism to measure planner underruns, which we know is the likely cause of print quality issues, we can attempt to mitigate the issue.

My previous investigation with Octoprint's comm.py which manages the communication with the printer indicated that the default behaviour is to wait for an ok from the printer before sending the next command, which means that the command buffer is usually empty by the time the next command reaches the printer.

Generally, the planner buffer is usually kept full and does not usually underrun for the scenarios I have tested (apart from curves on Cura 4.7.1), however any potential delay which could be introduced by e.g. CPU load, resends (noise on USB cable) – can cause planner buffer underruns.

Making Octoprint send multiple commands inflight

The core algorithm to keep the command buffers full is as follows:

• Check if the printer is reporting available capacity in the command buffer
• Trigger Octoprint to send more commands

So at the minimum, we need a way to detect the available capacity in the command buffers, and a mechanism to trigger Octoprint to send more commands. We know we can use Marlin's ADVANCED_OK output for undersanding the available buffer capacities, we need to figure out a mechanism to trigger sends.

Octoprint's core logic for sending commands is inside comm.py's _send_loop, which is running in a thread, and checks for _send_queue to have something to send, and once command has sent, waits for _clear_to_send.wait() which is a CountedEvent / mutex mechanism that lets other threads tell the thread that it's cool to send another command.

_clear_to_send.set() in another thread is ultimately what causes the next command to be sent, so it looked like a good mechanism to start.

I wanted to add this buffer-filling functionality as a plugin because I feel a tad uncomfortable with introducing significant changes to comm.py, so I began with a rough naive plugin that inspects the ADVANCED_OK output and calls _clear_to_send.set() if there's capacity.

Turns out, this was too naive as the plugin would react to responses that do not reflect the current state of the buffers, and it doesn't know which lines it triggered, so it would cascade into serial buffer overruns extremely quickly. I also discovered the ok buffer size that determines the maximum _clear_to_send can buffer needs to be at least 2, otherwise calling _clear_to_send.set() won't do anything if _clear_to_send is already at 1.

My next attempt would keep track the number of commands inflight and use this as a basis to determine whether or not _clear_to_send.set() should be called, as added a minimum delay between triggering a send, and this worked pretty well considering the amount of code required:

ADVANCED_OK = re.compile(r"ok (N(?P<line>\d+) )?P(?P<planner_buffer_avail>\d+) B(?P<cmd_buffer_avail>\d+)")
LINE_NUMBER = re.compile(r"N(?P<line>\d+) ")

class BufferMonitorPlugin(octoprint.plugin.StartupPlugin):
    # ok buffer must be above 1
    def __init__(self):
        self.bufsize = 4
        self.max_inflight = self.bufsize
        self.last_cts = time.time()
        self.last_sent_line_number = 0

    def on_after_startup(self):
        self._logger.info("Hello World!")

    def gcode_sent(self, comm, phase, cmd, cmd_type, gcode, *args, **kwargs):
        self.last_sent_line_number = comm._current_line    
            
    def gcode_received(self, comm, line, *args, **kwargs):
        if "ok " in line:
            matches = ADVANCED_OK.search(line)

            if matches.group('line') is None:
                return line

            line_no = int(matches.group('line'))
            buffer_avail = int(matches.group('cmd_buffer_avail'))
            inflight = self.last_sent_line_number - line_no

            if inflight > self.max_inflight:
                # too much in flight, scale it back a bit
                comm._clear_to_send.clear()
                self._logger.info("too much inflight, chill a bit")
                self._logger.info("Buffer avail: {} inflight: {} cts: {}".format(buffer_avail, inflight, comm._clear_to_send._counter))

            if buffer_avail >= 1 and (time.time() - self.last_cts) > 0.5 and inflight < self.max_inflight:
                self._logger.info("sending more")
                queue_size = comm._send_queue._qsize()
                self._logger.info("Buffer avail: {} inflight: {} cts: {} queue: {}".format(buffer_avail, inflight, comm._clear_to_send._counter, queue_size))
                self.last_cts = time.time()
                comm._clear_to_send.set()

        return line

I was able to confirm with buffer monitoring via M576 that it fills the buffers and decreases underruns. Graphs further below.

Note: I discovered that my Ender 3 v2 takes at minimum 9ms to respond to a command, where the timing was measured by inspecting the serial.log of Octoprint. This means that the default behaviour of waiting for an ok to send the next command was capped at ~111 commands a second, and I know that my Cura 4.7.1 sliced 3DBenchy will easily spike to 160 commands a second, which will most definitely cause underruns.

Making a plugin in Octoprint

Now that the core idea has proven to be workable, I began refining the plugin logic and testing with both dry-run prints and real prints to determine reliability.

I followed the Octoprint plugin guide and developed the plugin against a Docker container on my local machine for speed, but ran into different behaviour with the Virtual Printer that comes with Octoprint, so I could only really test the plugin against my Ender 3 v2 on my Octopi setup.

I called the plugin "BufferBuddy" after some deliberation cause the working title "buffer-filler.py" sounded a bit shit.

I ran into issues getting the plugin to load initially which eventually turned out to be a Python version specification issue. Once this was sorted, I was able to copy across the core logic from my prototype and began fleshing it out.

Understanding how to implement a UI took probably three times as long as implementing the core logic, which was hampered by needing to restart Octoprint to see changes. However, I eventually managed to make the UI behave the way I wanted!

Introducing BufferBuddy

It's probably easiest to explain the impact of the plugin with graphs.

The graphs below are from graphing M576 output during a print of a 50% scale 3DBenchy sliced using Cura 4.6.2 and 4.7.1, and with BufferBuddy enabled/disabled, with the leading underrun artifacts caused by the purge line removed for clarity.

My printer is an Ender 3 v2 running Smith3D's Marlin fork which has improvements for the Ender 3 v2's LCD, with my own patches for M576, with BUFSIZE=16, BLOCK_BUFFER_SIZE=16, USART_RX_BUF_SIZE=64, USART_TX_BUF_SIZE=64.

The graphs indicate that we consistently see command buffer underruns, but only see ~6 instances of planner buffer underruns, where the maximum detected period that the planner buffer was empty was under 50ms.

With BufferBuddy enabled, command buffer underruns are mostly eliminated, with 13 instances of command underruns during print, and one planner buffer underrun with a max underrun period of 9ms.

For Cura 4.7.1, which we know produces problematic gcode, it's another story.

It shows severe planner underruns, with delays easily going above 75ms. Commands per second easily surpasses 150 per second, which is above the max throughput of ~111 per second which we calculated by measuring command/ack latency above.

BufferBuddy eliminates most of the planner buffer underruns, but there is still command buffer underruns due to the sheer gcode density, although it halves the maximum delay the command buffers remain empty.

Uploading to SD appears to behave differently with respect to having multiple lines inflight for more throughput, as the command buffer never gets filled, and it seems to be more dependent on serial RX buffer size which we can't easily detect, so this needs more work.

Actual Print Quality

So, now we know that we've significantly mitigated planner underruns with BufferBuddy, we can now see if we've resolved print quality issues with Cura 4.7.1 on an actual print.

Turns out Octoprint can detect when I print from SD from the Ender 3's interface, so I was able to get a "control" print with best case scenario for buffer filling.

Printing from SD showed zero planner and command underruns during the print (tiny smidge at the end probably due to built-in print completion commands), but commands per second peaking above 300 shows that it's going to be extremely hard to keep buffers filled due to latency when it's super dense gcode.

Printing of SD is a little bit better in the middle, but still exhibits over-extrusion on curves. The motors seem to make odd sounds during these curves, which is likely related.

For comparison, this is a 3DBenchy from Cura 4.6.2:

Putting aside that iPhone cameras aren't amazing at capturing the detail I want (handheld, at least), the Cura 4.6.2 sliced Benchy looks pretty good by comparison still.

Summary.. so far

So, it looks like BufferBuddy doesn't fully address the Cura 4.7.1 problem, but it still mitigates against potential planner underruns by keeping the command buffers full, which still should help against the occasional blip of load on lower-powered devices.

Want to check out the plugin? It's on my Github, but it's still considered experimental and may cause your printer to lock up.

This is a sequel of my post about diagnosing 3D print quality when printing with Octoprint.

Now that we have a working theory for the reduced print quality, the next step was to know for sure when the problem was occurring. If I could have the printer tell me when it has no more instructions, it would be the first step to be able to objective measure how often the issue was occurring.

I dived into the source code for Marlin to begin understanding where I can hook into the relevant events, and what kind of metrics I could easily extract.

I consider myself mediocre with C/C++, and it's extremely rusty at best, and embedded C/C++ has its own shenanigans, so I may be wrong about how stuff works.

Understanding what to change

Marlin's core loop is as follows:

/**
 * The main Marlin program loop
 *
 *  - Call idle() to handle all tasks between G-code commands
 *      Note that no G-codes from the queue can be executed during idle()
 *      but many G-codes can be called directly anytime like macros.
 *  - Check whether SD card auto-start is needed now.
 *  - Check whether SD print finishing is needed now.
 *  - Run one G-code command from the immediate or main command queue
 *    and open up one space. Commands in the main queue may come from sd
 *    card, host, or by direct injection. The queue will continue to fill
 *    as long as idle() or manage_inactivity() are being called.
 */

I traced BUFSIZE to queue.cpp, where the logic for reading off serial comms and the command queue is handled. This is how I think it works:

• idle() (this seems poorly named?), calls manage_inactivity(), which in turns calls queue.get_available_commands() if there's enough room in GCodeQueue::command_buffer which has max BUFSIZE elements.
• queue.get_available_commands() pulls data from serial / SD card, performs basic parsing, checksum validation, early handling
• If it's all good and it doesn't need to handle it early, it chucks it in the command_buffer ring buffer via _enqueue, with say_ok flag for that index set to true.
• The say_ok flag is set in another array and not immediately sent to the host, which is not what I expected.
• Other core tasks are run, such as timer checks, UI, auto-reporting, etc
• Finally, queue.advance() is called, which invokes process_next_command(), which parses and executes the command
• process_parsed_command() is a behemoth of a switch statement, which figures out what function to run based on the gcode
• Once it runs the relevant function, by default, it will call queue.ok_to_send(), which then sends the ok back to the host that Octoprint is waiting for.

While trying to understand how the core loops worked, I spotted the ADVANCED_OK block that exposes the planner and command buffer capacity, which were planner.moves_free() and BUFSIZE - [queue.]length, which is a great start to report.

The problem with understanding buffer underruns with the ADVANCED_OK report, is it can only report the state of those buffers when the ADVANCED_OK is sent. We can infer if it returns B(BUFSIZE - 1)that the command buffer was empty before we sent the command, but we don't have much other information from this.

I considered adding more instrumentation to the ADVANCED_OK response, however it would increase serial comm load on both ends cause it's called on every command, so I figured I needed to add my own gcode that can report the data I wanted, optionally on an interval.

Implementing a buffer monitoring gcode

I looked at the existing gcode list to see if there was anything similar to what I wanted to expose already, but there didn't appear to any I could easily extend. I decided to use M576, as M575 was "Set baud rate", which was somewhat relevant to buffer monitoring.

I used the auto temperature reporting module as a base, and hooked into GCodeQueue::advance() for my logic. A few iterations (which annoyingly requires me to flash via microSD), I had a working M576 command that returns M576 P<nn> B<nn> U<nn>, where:

• P is planner buffer available
• B is command buffer available (both from ADVANCED_OK)
• U is number of command buffer underruns since last report

It also supported M576 S<n> where n is the number of seconds between automatic reports.

Testing the concept

I ran it through a dry-run version of a half-scale 3DBenchy gcode, where all the extrusion instructions were stripped out, and combined with a simple Octoprint plugin I had hacked together, I was able to observe the output of my newly minted gcode command!

Initial findings showed that there was many underruns when printing through Octoprint, anywhere from 5-30 per second. I realised it's not the number of buffer underruns that would be the issue, but how long the queue was in an empty state.

Iterating and adding more metrics

I added M<nn> to represent the maximum time in milliseconds that the command buffer was empty between commands. This resulted in much more useful information regarding on how long Marlin may be waiting for a command.

However, I noticed when the max buffer empty time was as high as 100ms, the planner buffer generally remained full, which would mean that the printer would still have movement queued and thus not actually manifest in stalled motion.

I decided to also add planner buffer underrun metrics. This was more difficult to try to figure out where to make the change, as ideally we would detect if the queue was empty immediately after it was to be processed, however this logic was in a dedicated stepper ISR (some kind of interrupt handler) and it seemed like a bad idea to modify that, so I settled for hooking it into the auto_report_buffer_statistics that runs in a fairly tight loop.

I changed the output to the following:

 * When called, printer emits the following output:
 * "M576 P<nn> B<nn> PU<nn> PD<nn> BU<nn> BD<nn>"
 * Where:
 *   P: Planner buffers available 
 *   B: Command buffers available
 *   PU: Planner buffer underruns since last report
 *   PD: Maximum time in ms planner buffer was empty since last report
 *   BU: Command buffer underruns since last report
 *   BD: Maximum time in ms command buffer was empty since last report

Now we can tell when and how long the motion planner buffer was empty for!

Testing methodology

To compare, I sliced a half-scale 3DBenchy with the same custom profile in both Cura 4.6.2 and Cura 4.7.1, as I know from personal experience that Cura 4.7 introduced a bug where it generates extremely dense gcode around curves.

Cura 4.7.1 generates as much as double(!!) the amount of gcode for the same model and settings, which definitely will cause issues if Octoprint is unable to stream the gcode to the printer to be printed at the speed that it was designed to be printed at.

I ran it through my dry-run.rb and "printed" these with M576 reporting every 2 seconds and chucked it into a log file for later processing.

I processed the logs into CSV and chucked it into Numbers to visualise the difference.

The results

The key metric here is Planner Max Empty time, represented in red and is in milliseconds, followed by Planner Underruns in yellow as a count.

In Cura 4.6.2, we see only 9 instances of planner buffer underruns, and the max time the buffer was empty was 36ms.

However, 4.7.1 is a whole 'nother story, with 200+ planner buffer underruns.

I'm not sure of the threshold where the motion planner buffer being empty results in visible print artifacts, but I feel like anything over 50ms is noticeable. Theoretically it should be able to test this by injecting pauses but that's for another time.

Next steps

Now that we have the ability to have some hard data when the motion planner buffer underruns, we can now attempt to address the issue and measure if it helped or not.

I'll be opening a pull request for the M576 gcode into Marlin.

Stay tuned for the next one!

Update: See my post on Adding buffer monitoring to Marlin.

I recently picked up a Creality Ender 3 v2 to finally check out 3D printing. Initially, I was very impressed with the quality of my prints for a budget printer, however, at some point during tinkering, I noticed zits on my prints, specifically around curves.

I eventually tracked this down to a combination of printing over USB in Octoprint vs printing from SD card, as well as switching from Creality's slicer (Cura 4.2) to Cura 4.7.1.

A bug (#8321) was introduced in Cura 4.7 where it was adding loads of tiny segments in curves, which in turn generates way more gcode for the same curve. This, combined with how Octoprint streams the gcode over the USB serial connection to the printer, results in the printer's buffers to empty, thus causing brief pauses as it waits for more gcode, which manifests in zits on the surface of prints as the pressure in the nozzle still causes extrusion to occur.

Even though printing via SD card is likely to resolve the issue, it removes a lot of the convenience and power of printing with Octoprint, such as the ability to selectively cancel certain regions of your print, saving time and reducing waste. Cura's segment issue is likely to be addressed at some point, which will decrease the likelihood of reduced print quality due to less gcode, however the issue with Octoprint's gcode streaming still exists, and has been reported since 2014.

What causes print artifacts when printing over USB?

There are a couple of factors at play that can affect the streaming of gcode to the printer when using Octoprint, especially on embedded-class devices like Raspberry Pis.

• CPU load: If the process in Octoprint that's responsible for streaming gcode doesn't get to run due to load on the system, then the printer is likely to be waiting for additional instructions, causing movement stutter and print artifacts. This can occur when just loading the Octoprint interface. There is a reason why stuff like plugin management and timelapse processing is prevented while a print is in progress.
• USB cable: I was using a longer cable I had lying around which occasionally would have disconnect issues, which was extremely noticable when I experimented with Klipper (more below). Switching to a shorter cable resolved this particular issue.
• Communication speed: It's possible that the gcode is simply too dense for it to be communicated over the USB/serial connection at a rate that it's needed to be processed by the printer to perform the print as desired.

Mitigations

After learning the issue is likely to do with gcode density around curves, I tried to use the ArcWelder plugin which makes gcode smaller by converting the straight G0/G1 commands into G2/G3 arc commands, which can reduce the resulting gcode by as much as 80%!

However, the stock firmware I was using did not have ARC_SUPPORT enabled, so I gave Klipper a shot. The idea of Klipper is to move the motion calculation off the printer's (usually underpowered) microcontrollers, and on to a more powerful device like a Raspberry Pi (compared to a microcontroller, of course). It supplies its own firmware for the printer, which takes a compressed data stream that in turn tells it how to do what to its various stepper motors. Klipper then exposes its own serial port to Octoprint which receives gcode commands.

Using Klipper noticably improved print quality for me, even when using Cura 4.7.x, however the printer display was just blank and not being able to control the printer with its built-in controls was a significant negative. I also ran into severe extruder chattering which caused filament grinding when I was using its pressure advance feature to handle better corners.

Marlin has a DIRECT_STEPPING option which is similar to how Klipper works, however it recommends 250k-500k baud and it doesn't seem to be used very much just yet.

My BLTouch bed levelling sensor eventually arrived so I followed Smith3D Ender 3 v2 BLTouch guide, and used their fork of Marlin which has some nice improvements.

I downgraded Cura to 4.6.2 for the time being, which resolved most of my print quality issues.

However, the core issue of potential print artifacts due to printing over USB still remain, and I'm not willing to give up the convenience of Octoprint, so I investigated further.

Ideally, the code responsible for streaming to the printer should be run at a much higher priority to ensure it gets scheduled. It appears Octoprint's sending_thread in comm.py is being run as a daemon thread, which I initially got excited cause it sounded like it would run it as a separate process (somehow?) which means we could just snice it to a higher priority, but that's not what it does.

My limited understanding of comm.py seems to indicate that Octoprint sends the next command to the printer once it's received an ok from the printer, which happens when Marlin commits the command into its ring buffer. The size of the ring buffer is defined by BUFSIZE, and it's usually set to 4 for most configurations.

This seems pretty small to me, but if Octoprint isn't filing the buffer reliably, then increasing BUFSIZE should decrease the likelihood but not mitigate it completely.

There is a WIP pull request that enables parsing ADVANCED_OK to fill buffers accordingly, however it was last touched September 2019, so it may never get merged in.

Detecting the issue

We should be able to discover the minimum speed that we can communicate reliably to the printer via a benchmark, and in theory, we should be able to look at a gcode file and determine if there are parts of the gcode that cannot be transmitted at the speed that it's meant to be parsed at.

There is also an ADVANCED_OK configuration option which will report the line number of the command being acked, as well as the remaining command buffer and motion planning buffers.

I'm in the progress of writing an Octoprint plugin which should be able to track these buffers, as well as hopefully calculate a median latency of command to response to further diagnose this issue.

Ideally, there should be a way to detect when the command buffer is empty and report back to the host so users can be aware that there are issues happening. Initial research shows that void GCodeQueue::advance() is where the change should be made.

Next up: Adding buffer monitoring to Marlin.