Nick Whyte

Ender 3 v2 Pen Plotter

Mon, 25 Apr 2022 00:00:00 +0000

Last year I purchased myself an Ender 3 V2 3D Printer. It was a logical purchase, to compliment my home automation hobby, allowing me to design and print custom enclosures for miscellaneous ESPHome and Zigbee nodes.

It turns out having a CoreXY 3D printer allows you to do more than just 3D printing if you’re willing to get creative. For example, adding on a laser engraver.

In my case, I was tasked with automating the process of drawing an embroidery pattern onto fabric using a heat/friction erasable pen for my partner, Kate.

Unsurprisingly, there are services available which can do this, albeit using machine washable dye rather than heat reactive dye in most cases. However, in almost every case these are cost prohibitive for hobby embroidery projects and also happen to incur a long turn around time from order to delivery.

So as with most projects I undertake, rather than re-designing from scratch, I opted to start searching for existing projects. Surely someone had done this before! I managed to find a handful of models (e.g. Ender 3 v2 Pen Plotter addon ), however most of these either required you to fully remove the hotend assembly or did not accomodate mounting a BL-Touch levelling probe.

Fortunately I managed to find this model which both accommodated both of my requirements. It supported mounting of the BL-Touch probe without needing to remove the hotend assembly. I printed it and mounted it to the printer… job done… right? Unfortunately not. Due to the way it mounted it now meant that the hot-end carriage could no longer hit the x-axis limit switch. Not good!

From here, the only logical solution was to design my own, which accommodated for my specific requirements. I’ve been getting more and more familiar with Fusion 360 since buying the printer, so was confident to get straight to prototyping a design.

I started off by importing dimensionally accurate models of both the Ender 3 v2 print head assembly/x-carriage and the BL-Touch:

By doing so, I would be able to project the precise positions of mounting holes onto the sketches for component of my attachment, rather than measuring by hand with some amount of inaccuracies.

Another requirement I came to realise was that it would be useful would be the ability to remove the plotter attachment when it is not in use. As such I arrived at my first design iteration:

The design was relatively simple. Two main parts; a mounting arm to hold the BL-Touch probe to the hotend assembly, and a three-piece arm to hold the pen, which is mounted to the mounting arm using the same screws from the BL-Touch probe. The three-piece arm was designed to allow a pen to be mounted with some spring tension, however in actual fact once it was printed there was too much friction to provide any reasonable amount of travel. Back to the drawing board.

On to the next iteration, I re-thought the design. I reused the same mounting arm to the hotend assembly, but changed the mounting angle of the pen attachment. Instead, it would side-mount to the mounting arm using M3 bolts into heat-set inserts. I thought I would give this a try, without any tension mechanism, as an automatically leveled bed should suffice.

One printed, I came to realise that the side mounting again caused some interference with the x-axis limit switch, so I found that in order to solve this problem I needed to move the rear bolt closer to the front, and additionally provide a countersunk for the bolt head to sit inside.

After a handful more prints and some more tweaks to the height of mounting arm and hole sizes, I had a final design:

Now, with the mounting bracket complete, it was time to move onto the software aspect of the project. Fortunately for me, this is a solved problem. Uri Shaked has written an excellent post about using Inkscape with the Gcodetools extension to produce plotting toolpats from SVG files. Inkscape isn’t the nicest software to run on MacOS, so the experience is a little clunky (and crashy), but it does the job. However, I plan on exploring other tools, like Inkcut, JScut, juicy-gcode, SVG-toGcode and hp2xx in due time.

Gcodetools allows you to specify a custom header and footer for produced gcode files. I used this to provide a more ergonomic printing experience by ensuring the printer was homed, bed levelling mesh is enabled, and the print timer is initiated to allow for on-the-fly tweaking of the z-height.

Header:

M420 S1 ; Enable Jyers Mesh
M75 ; Start print timer
G28 ; home all axis

Footer:

G1 Z20.00 F600 ; Move print head up
G1 X5 Y180 F9000 ; present print
M84 X Y E ; disable motors
M77 ; Stop print timer

Now, with some G-code produced, it was time for a test. As a software engineer, there’s no better way to test the output capability of something than to print("hello world"), and so that is exactly what I did:

Great success! From here, I did some further tests on fabric and found it worked just fine, albeit a little faint since more pen pressure resulted in snagging the fabric, so for fabric prints 2 or 3 passes is generally necessary. Beyond just plotting SVGs I’ve started to explore generative art using mathematical functions (like a spirograph). Here are a few pictures from testing and a video:

You can find the model for my Ender 3 V2 BL Touch Mount and Pen Plotter on Thingiverse and Thangs

100 Warm Tunas 2018 Prediction Analysis

Mon, 04 Feb 2019 00:00:00 +0000

Over the space of 6 weeks, 100 Warm Tunas collected a large sum of data and chugged away at it to make some predictions about what the Hottest 100 of 2018 would look like.

In summary,

We collected 6,234 entries (13.6% decrease since 2017 🔻).
We tallied 58,463 votes across these entries (12.9% decrease since 2017 🔻).
3.00% of votes were collected via Instagram direct message.
Triple J counted 2,758,584 votes.
Therefore, we collected a sample of 2.12%.
We successfully predicted #1
We predicted 7 out of the top 10 songs.
We predicted 15 out of the top 20 songs.
We predicted 83 out of the 100 songs played in the countdown.
Throughout December and January, 100warmtunas.com was loaded over 105,000 times by over 28,000 users.

You can read the full technical prediction analysis over at the 100 Warm Tunas news site.

100 Warm Tunas has a new home!

Thu, 01 Nov 2018 00:00:00 +0000

100 Warm Tunas has found a new home this year! This year’s results, along with the past 2 years can now be found at 100warmtunas.com. The existing domain (100-warm-tunas.nickwhyte.com) will simply perform a 301 permanent redirect to the new domain, so all existing inbound links should be unaffected.

100 Warm Tunas 2017 Prediction Analysis

Fri, 16 Feb 2018 00:00:00 +0000

🔥💯

Over the space of 6 weeks, 100 Warm Tunas collected a large sum of data and chugged away at it to make some predictions about what the Hottest 100 of 2017 would look like. Along the way we encountered a bug in the collection process, however data was backfilled and showed that I had collected a sample size around the same as in 2016.

Summary

100 Warm Tunas collected 7,216 entries (7.3% less than 2016 🔻)
100 Warm Tunas tallied 67,085 votes across these entries (2.6% more than 2017 🔺). This is due to improvements in 100 Warm Tunas’ counting and recognition process.
Triple J counted 2,386,133 votes.
Therefore, 100 Warm Tunas, collected a sample of 2.8%. Not bad! (The same as in 2016).
Warm Tunas predicted 8 out of the top 10 songs (Same as 2016) (Ignoring order)
Warm Tunas predicted 16 out of the top 20 songs (3 less than in 2016, where 19 out of 20 were predicted) (Ignoring order).
Warm Tunas predicted 83 out of the 100 songs played in the countdown. (1 less than in 2016) (Ignoring order)

Overall, even though the sample size was reasonably consistent between 2016 and 2017, it is clear that the results collected in 2016 were more accurate.

Technical Analysis

The results this year definitely show a more accurate 1st place prediction (predicting HUMBLE. to win), as opposed to last year where the top two positions were placed out of order, however looking at the data, it looks as though all other aspects of the prediction stayed almost the same.

To start this analysis, lets take a look at the top 10 of the official countdown and match it up with their predicted places in Warm Tunas:

Artist	Title	ABC Rank	Tunas Rank	Difference
Kendrick Lamar	HUMBLE.	1	1	0
Gang Of Youths	Let Me Down Easy	2	3	1
Angus & Julia Stone	Chateau	3	6	3
Methyl Ethel	Ubu	4	4	0
Gang Of Youths	The Deepest Sighs, The Frankest Shadows	5	2	3
Lorde	Green Light	6	8	2
PNAU	Go Bang	7	5	2
Thundamentals	Sally {Ft. Mataya}	8	10	2
Vance Joy	Lay It On Me	9	15	6
Gang Of Youths	What Can I Do If The Fire Goes Out?	10	13	3
BROCKHAMPTON	SWEET	11	7	4
Peking Duk & AlunaGeorge	Fake Magic	12	16	4
Khalid	Young Dumb & Broke	13	24	11
Lorde	Homemade Dynamite	14	30	16
Vera Blue	Regular Touch	15	11	4
Jungle Giants, The	Feel The Way I Do	16	32	16
Baker Boy	Marryuna {Ft. Yirrmal}	17	12	5
Ball Park Music	Exactly How You Are	18	14	4
Killers, The	The Man	19	19	0
Peking Duk	Let You Down {Ft. Icona Pop}	20	38	18

Lets pull apart this table and grab some statistics about how we did with our prediction:

Predicted	Out Of Top N	Percentage
8	10	80.0%
16	20	80.0%
22	30	73.3%
33	40	82.5%
42	50	84.0%
50	60	83.3%
62	70	88.6%
68	80	85.0%
78	90	86.7%
83	100	83.0%

So from the above data, it’s apparent that once again:

The average error for the top ten ranks was 2.2 positions (an increase from 2016’s 1.9 positions)
Warm Tunas predicted 8 out of the top 10 songs
Warm Tunas predicted 16 out of the top 20 songs
Warm Tunas predicted 83 out of the 100 songs played in the countdown.

That’s not a bad result at all!

The average rank prediction error, grouped into divisions of 10 is provided below. It shows that it’s difficult to predict where songs will place once you leave the top 50:

ABC Position	Warm Tunas Avg Error
1-10	1.9000
11-20	8.2000
21-30	14.3000
31-40	12.5000
41-50	15.2000
51-60	24.7000
61-70	18.2000
71-80	29.9000
81-90	34.1000
91-100	29.5000

To compare Warm Tuna’s predictions vs actual rankings, a scatter plot has been provided below. We can see as we get closer to rank 1, the 100 Warm Tunas prediction gets better and converges upon the actual rankings played out on the day.

Fortunately this year around, 100 Warm Tunas was able to successfully predict the winner of the countdown. The reason this prediction was able to be made was because the sample collected clearly indicated HUMBLE. as an outlier. – an entire 5% higher than the next track, predicted to place 2nd.

Anyway, that’s a wrap. See you later this year for 100 Warm Tunas 2018 edition!

100 Warm Tunas 2017 Update 🔥💯

Tue, 23 Jan 2018 00:00:00 +0000

100 Warm Tunas has been happily chugging away for the last month or so. I’ve obtained a fair amount of media coverage too.

A couple of days back, I posted the site to the triplej subreddit. Someone replied to the post telling me my vote count was significantly less than what they had been counting by hand, which made me somewhat suspicious – was there a bug in my Instagram scraping library that I built?

Well, after a bit of debugging early this morning, I found that there was indeed a bug. Not a bug with my scraping library, but rather a bug with how I was using the library:

-    for page in ig.fetch_pages('triplej', per_page=10):
+    for page in ig.fetch_pages(hashtag, per_page=10):
         for post in page.posts:
             if post.is_video:
                 logger.info("Skipping {} because it's a video".format(post.shortcode))

For those who are programmers, you’ll probably spot the issue here. For those who aren’t, the issue is that I have been using a hardcoded string to collect Instagram votes, when I thought I was collecting a handful of hashtags.

This has now been rectified and I have kicked off a full re-scrape to back-fill the data.

100 Warm Tunas 2017

Sun, 17 Dec 2017 00:00:00 +0000

🔥💯

Last year I predicted the top 3 in Triple J’s hottest 100 (ignoring order). This year I’m back at it once again with an updated webpage and a Spotify playlist.

Results are collected, optimised, and processed multiple times per day. Instagram images tagged with #hottest100 and a few others are included for counting.

Happy voting!

Feel free to check out the results from 2016, 2016 results analysis, and the process taken in 2015.

100 Warm Tunas 2016 Prediction Analysis

Sun, 17 Dec 2017 00:00:00 +0000

It’s been a long time since the Hottest 100 of 2016 was aired. Unfortunately, I never really got around to publishing some analysis I performed on the prediction results. Fortunately, I managed to find some time recently!

Looking from afar, the results don’t look fantastic (when you compare them to my results from 2015 at least). The prediction unfortunately predicted the top two places out of order, however did manage to predict the third place correctly.

Lets take a look at the Top 10 of Triple J’s list and match it up with 100 Warm Tunas:

Looking at this we see most predictions we can find some learnings:

The average error for the top ten rank was 1.9 rank positions.
If 100 Warm Tunas ignored rank and simply guessed the top ten, it would have predicted 8 of the top 10 songs.
If 100 Warm Tunas ignored rank and simply guessed the top 3 songs to win, it would have predicted all 3 songs. Woo!

Lets dive into a chart that shows error for all ranks:

From this chart, we can deduce that the further away from position 1 we become, the higher the error. This information alone isn’t very useful. We can get a better understanding of error by finding the average for each ranking group:

As we get closer to rank 1, the results become more and more accurate, however they are not perfect. This is more obvious if we use a scatter plot to compare Triple J ranks against Warm Tunas predictions:

It’s clear now that as we get closer to rank 1, the 100 Warm Tunas prediction gets better and converges upon the actual rankings played out on the day. However, unfortunately this year the difference between rank 1 and rank 2 was way too close to call - just 0.67% of voting volume was separating the two. A difference that was not enough to provide an accurate prediction of the winner.

Overall, whilst 100 Warm Tunas 2016 did get the two top positions out of order, it’s understandable as to why this happened. Hopefully this year there is a greater difference between ranks, giving further ability to predict the winner in position #1.

Reverse Engineering a 433MHz Motorised Blind RF Protocol

Sat, 15 Jul 2017 00:00:00 +0000

I’ve been doing a fair bit of DIY home automation hacking lately across many different devices - mostly interested in adding DIY homekit integrations. A couple of months ago, my dad purchased a bulk order of RAEX 433MHz RF motorised blinds to install around the house, replacing our existing manual roller blinds.

Note: If you are based in Australia, you can purchase these in bulk or individually via www.raexaustralia.com (Full disclosure – my father runs the site).

The blinds are a fantastic addition to the house, and allow me to be super lazy opening/closing my windows, however in order to control them you need to purchase the RAEX brand remotes. RAEX manufacture many different types of remotes, of which, I have access to two of the types, depicted below:

R Type Remote (YRL2016)

X Type Remote (YR3144)

Having a remote in every room of the house isn’t feasible, since many channels would be unused on these remotes and thus a waste of $$$ purchasing all the remotes. Instead, multiple rooms are programmed onto the same remote. Unfortunately due to this, remotes are highly contended for.

An alternate solution to using the RAEX remotes is to use a piece of hardware called the RM Pro. This allows you to control the remotes via your smartphone using their app

The app is slow, buggy and for me, doesn’t fit well into the home-automation ecosystem. I want my roller blinds to be accessible via Apple Homekit.

In order to control these blinds, I knew I’d need to either:

Reverse engineer how the RM Pro App communicated with the RM Pro and piggy-back onto this
Reverse engineer the RF protocol the remotes used to communicate with the blinds.

I attempted option 1 for a little while, but ruled it out as I was unable to intercept the traffic used to communicate between the iPhone and the hub. Therefore, I began my adventure to reverse engineer the RF protocol.

I purchased a 433MHz transmitter/receiver pair for Arduino on Ebay. In case that link stops working, try searching Ebay for 433Mhz RF transmitter receiver link kit for Arduino.

Initial Research

A handful of Google searches didn’t yield many results for finding a technical specification of the protocol RAEX were using.

I could not find any technical specification of the protocol via FCC or patent lookup
Emailed RM Pro to obtain technical specification; they did not understand my English.
Emailed RAEX to obtain technical specification; they would not release without confidentiality agreement.
I did find that RFXTRX was able to control the blind via their BlindsT4 mode, which appears to also work for Outlook Motion Blinds.
After opening one of the remotes and identifying the micro-controllers in use, I was unable to find any documentation explaining a generic RF encoding scheme being used.
It may have been possible to reverse engineer the firmware on a remote by taking an I2C dump of the ROM chip. It seems similar remotes allow dumping at any point after boot

Capturing the data

Once my package had arrived I hooked up the receiver to an Arduino and began searching for an Arduino sketch that could capture the data being transmitted. I tried many things that all failed, however eventually found one that appeared to capture the data.

Once I captured what I deemed to be enough data, I began analysing it. It was really difficult to make any sense of this data, and I didn’t even know if what had been captured was correct.

I did some further reading and read a few RF reverse engineering write-ups. A lot of them experimented with the idea of using Audacity to capture the signal via the receiver plugged into the microphone port of the computer. I thought, why not, and began working on this.

This captures a lot of data. I captured 4 different R type remotes, along with 2 different X type remotes, and to make things even more fun, 8 different devices pairings from the Broadlink RM Pro (B type).

From this, I was able to determine a few things

The transmissions did not have a rolling code. Therefore, I could simply replay captured signals and make the blind do the exact same thing each time. This would be the worst-case scenario if I could not reverse engineer the protocol.
The transmissions were repeated at least 3 times (changed depending on the remote type being used)

Zooming into the waveform, we can see the different parts of a captured transmission. This example below is the capture of Remote 1, Channel 1, for the pairing action:

Zooming in:

In the zoomed image you can see that the transmission begins with a oscillating 0101 AGC pattern, followed by a further double width preamble pattern, followed by a longer header pattern, and then by data.

This preamble, header and data is repeated 3 times for R type remotes (The AGC pattern is only sent once at the beginning of transmission). This can be seen in the first image.

Looking at this data won’t be too useful. I need a way to turn it digital and analyse the bits and determine some patterns between different remotes, channels and actions.

Decoding the waveform.

We need to determine how the waveform is encoded. It’s very common for these kinds of hardware applications to use one of the following:

Manchester Encoding,
Tri-State/Tri-bit Encoding, Additional info
PWM Encoding
Raw? high long = 11, high short = 1, low long = 00, low short = 0?

By doing some research, I was able to determine that the encoding used was most likely manchester encoding. Let’s keep this in mind for later.

Digitising the data

I began processing the data as the raw scheme outlined above (even though I believed it was manchester). The reason for this is that if it happened to not be manchester, I could try decode it again with another scheme. (Also writing out raw by hand was easier than doing manchester decoding in my head).

I wrote out each capture into a Google Sheets spreadsheet. It took about 5 minutes to write out each action for each channel, and there were 6 channels per remote. I began to think this would take a while to actually get enough data to analyse. (Considering I had 160 captures to digitise)

I stopped once I collected all actions from 8 different channels across 2 remotes. This gave me 32 captures to play with. From this much data, I was able to infer a few things about the raw bits:

Some bits changed per channel
Some bits changed per remote.
Some bits changed seemingly randomly for each channel/remote/action combination.
- Could this be some sort of checksum?

I still needed more data, but I had way too many captures to decode by hand. In order to get anywhere with this, I needed a script to process WAV files I captured via Audacity. I wrote a script that detected headers and extracted data as its raw encoding equivalent (as I had been doing by hand). This script produced output in JSON so I could add additional metadata and cross-check the captures with the waveform:

[
  {
    "filename": "/Users/nickw/Dropbox/RF_Blinds/Export_Audio2/tracks2/R1_CH1.wav",
    "captures": [
      {
        "data": "01100101100110011001100101101001011010010110011010011010101010101010101010011001101010101010101010101010101",
        "header_pos": 15751,
        "preamble_pos": 15071
      },
      {
        "data": "01100101100110011001100101101001011010010110011010100110101010101001101010011001101010101010101010101010101",
        "header_pos": 46307,
        "preamble_pos": 45628
      },
      {
        "data": "01100101100110011001100101101001011010010110011010010110101010101010011010011001101010101010101010101010101",
        "header_pos": 73514,
        "preamble_pos": 72836
      },
      {
        "data": "01100101100110011001100101101001011010010110011010101010101010100101010101101001011010101010101010101010101",
        "header_pos": 103575,
        "preamble_pos": 102895
      }
    ]
  }
]

Once verified, I tabulated this data and inserted it into my spreadsheet for further processing. Unfortunately there was too many bits per capture to keep myself sane:

I decided it would be best if I decoded this as manchester. To do this, I wrote a script that processes the raw capture data into manchester (or other encoding types). Migrating this data into my spreadsheet, it begins to make a lot more sense.

Looking at this data we can immediately see some relationship between the bits and their purpose:

6 bits for channel (C)
2 bits for action (A)
6 bits for some checksum, appears to be a function of action and channel. F(A, C)
- Changes when action changes
- Changes when channel changes.
- Cannot be certain it changes across remotes, since no channels are equal.
1 bit appears to be a function of Action F(A)
1 bit appears to be a function of F(A), thus, G(F(A)). It changes depending on F(A)’s value, sometimes 1-1 mapping, sometimes inverse mapping.

After some further investigation, I determined that for the same remote and channel, for each different action, the F(A, C) increased by 1. (if you consider the bits to be big-endian.).

Looking a bit more into this, I also determined that for adjacent channels, the bits associated with C (Channel) count upwards/backwards (X type remotes count upwards, R type remotes count backward). Additionally F(C) also increases/decreases together. Pay attention to the C column.

From this, I can confirm a relationship between F(A, C) and C, such that F(A, C) = F(PAIR, C0) == F(PAIR, C1) ± 1. After this discovery, I also determine that there’s another mathematical relationship between F(A, C) and A (Action).

Making More Data

From the information we’ve now gathered, it seems plausible that we can create new remotes by changing 6 bits of channel data, and mutating the checksum accordingly, following the mathematical relationship we found above. This means we can generate 64 channels from a single seed channel. This many channels is enough to control all the blinds in the house, however I really wanted to fully decode the checksum field and in turn, be able to generate an (almost) infinite amount of remotes.

I wrote a tool to output all channels for a seed capture:

./remote-gen generate 01000110110100100001010110111111111010101
...

My reasoning behind generating more data was that maybe we could determine how the checksum is formed if we can view different remotes on the same channel. I.e. R0CH0, R1CH0, X1CH0, etc…

Essentially what I wanted to do was solve the following equation’s function G:

F(ACTION_PAIR, CH0) == G(F(ACTION_PAIR, CH0))

However, looking at all Channel 0’s PAIR captures, the checksum still appeared to be totally jumbled/random:

Whilst looking at this data, however, another pattern stands out. G(F(A)) sits an entire byte offset (8 bits) away from F(A). Additionally the first 2 bits of F(A, C) sit at the byte boundary and also align with A (Action). As Action increases, so does F(A, C). Lets line up all the bits at their byte boundaries and see what prevails:

Colours denoting byte boundaries

Aligned boundaries

From here, we need to determine some function that produces the known checksum based on the first 4 bytes. Initially I try to do XOR across the bytes:

Not so successful. The output appears random and XOR’ing the output with the checksum does not produce a constant key. Therefore, I deduce the checksum isn’t produced via XOR. How about mathematical addition? We’ve already seen some addition/subtraction relationship above.

This appeared to be more promising - there was a constant difference between channels for identical type remotes. Could this constant be different across different type remotes because my generation program had a bug? Were we not wrapping the correct number of bits or using the wrong byte boundaries when mutating the channel or checksum?

It turns out that this was the reason 😑.

Solving the Checksum

Looking at the original captures, and performing the same modulo additions, we determine the checksum is computed by adding the leading 4 bytes and adding 3. I can’t determine why a 3 is used here, other than RAEX wanting to make decoding their checksum more difficult or to ensure a correct transmission pattern.

I refactored my application to handle the boundaries we had just identified:

type RemoteCode struct {
    LeadingBit uint // Single bit
    Channel    uint8
    Remote     uint16
    Action     uint8
    Checksum   uint8
}

Looking at the data like this began to make more sense. It turns out that F(A) wasn’t a function of A (Action), it was actually part of the action data being transmitted:

type BlindAction struct {
    Name  string
    Value uint8
}

var validActions = []BlindAction{
    BlindAction{Value: 127, Name: "PAIR"},
    BlindAction{Value: 252, Name: "DOWN"},
    BlindAction{Value: 253, Name: "STOP"},
    BlindAction{Value: 254, Name: "UP"},
}

Additionally, the fact there is a split between channel and remote probably isn’t necessary. Instead this could just be an arbitrary 24 bit integer, however it is easier to work with splitting it up as an 8 bit int and a 16 bit int. Based on this, I can deduce that the protocol has room for 2^24 remotes (~16.7 million)! That’s a lot of blinds!

I formally write out the checksum function:

func (r *RemoteCode) GuessChecksum() uint8 {
    return r.Channel + r.Remote.GetHigh() + r.Remote.GetLow() + r.Action.Value + 3
}

Additional Tooling

My remote-gen program was good for the purpose of generating codes using a seed remote (although, incorrect due to wrapping issues), however it now needed some additional functionality.

I needed a way to extract information from the captures and verify that all their checksums align with our rule-set for generating checksums. I wrote an info command:

./remote-gen info 00010001110001001101010111011111101010100 --validate
Channel:    196
Remote:     54673
Action:     STOP
Checksum:          42
Guessed Checksum:  42

Running with --validate exits with an error if the guessed checksum != checksum. Running this across all of our captures proved that our checksum function was correct.

Another piece of functionality the tool needed was the ability to generate arbitrary codes to create our own remotes:

./remote-gen create --channel=196 --remote=54654 --verbose
00010001101111110101010111111111010011001    Action: PAIR
00010001101111110101010110011111101101000    Action: DOWN
00010001101111110101010111011111111101000    Action: STOP
00010001101111110101010110111111100011000    Action: UP

I now can generate any remote I deem necessary using this tool.

Wrapping Up

There you have it, that’s how I reverse engineered an unknown protocol. I plan to follow up this post with some additional home-automation oriented blog posts in the future.

From here I’m going to need to build my transmitter to transmit my new, generated codes and build an interface into homekit for this via my homebridge program.

You can view all the work related to this project in the nickw444/homekit/blindkit repo.

As mentioned above, if you are based in Australia, you can purchase these blinds and associated accessories in bulk or individually via www.raexaustralia.com (Full disclosure – my father runs the site)

100 Warm Tunas 2016

Wed, 04 Jan 2017 00:00:00 +0000

🔥💯

Last year I predicted the top 3 results in order in Triple J’s hottest 100. This year I’m back at it again, however, now with a webpage and a Spotify playlist.

Results are collected, optimised, and processed multiple times per day. Instagram images tagged with #hottest100 and a few others are included for counting.

Happy voting!

You can read about the process last year here. However, vote collection is a fair bit more accurate this year.

Press:

A computer science nerd has predicted triple j’s Hottest 100 results (inthemix)
A Computer Science Grad May Have Figured Out How To Predict The Hottest 100 (Junkee)
This Dude’s Hottest 100 Prediction Method Is Pretty Damn Good (Stoney Roads)
We Could Be Looking At One Of The Biggest Hottest 100 Upsets In Years (Tonedeaf)
Has the Triple J Hottest 100 2016 Result Been Predicted? (Chattr)
Oddschecking The Triple J Hottest 100 (Oddschecker)
Triple J Hottest 100: Computer geek predicts the 2016 countdown (The New Daily)
Amy Shark Opens Up On The Pressure Of Being A Hottest 100 Favourite (The Music)
Who will win triple j’s Hottest 100 for 2016? (The Standard)
Triple J Hottest 100 crib notes: up-and-coming artists set to sweep the countdown (The Guardian)
Do We Already Know The “Hottest 100” Winner? (Student Edge)
Hottest 100 Experts On Who They Predict Will Top This Year’s Countdown (Music Feeds)
Could Amy Shark Or Flume Reach No 1 On This Years Hottest 100 (Daily Telegraph)

Other Mentions:

Edit: Woohoo, the Spotify playlist now has just over 1200 followers, and the website has had over 30,000 hits! That’s massive, thanks everyone!

Understanding and Tweaking some GPX data

Sat, 29 Oct 2016 00:00:00 +0000

As a casual bike rider, I enjoy tracking my rides with Strava so I can take a look at how my ride went and how well I performed throughout.

However, very rarely the Strava tracking application randomly crashes, or gets killed by iOS on my phone, during the ride. This means that the data was never recorded between the point at which the app died and the point when I became aware the app had died.

If we plot this type of failure, it looks something like this:

Fortunately in this case, there wasn’t too much missing data. However, I was still determined to learn about the GPX format and see if I could patch up the GPX file programatically.

In the specific case of the above map, I was riding north west, and at a point Strava crashed. Between this point and when I pulled out my phone to check my progress, no points were plotted. Google maps interprets this lack of data as a straight line between to the 2 points (as per GPX specification).

If we crack open the GPX file and take a look, we can see exactly what this looks like:

...
<trkpt lat="-33.9014420" lon="151.1066810">
    <ele>6.6</ele>
    <time>2016-10-28T23:38:50Z</time>
</trkpt>
<trkpt lat="-33.8802920" lon="151.0702190">
    <ele>20.9</ele>
    <time>2016-10-28T23:51:14Z</time>
</trkpt>
...

In it’s simplest form, a GPX file is an XML document that contains a sequence of GPS points (with associated metadata like elevation, and other depending on the tracker). This makes it reasonably simple for us to get our hands dirty and begin fixing the data set.

In order to add the missing data back into the GPX file, we need 3 things:

The last coordinate recorded before the app crashed
The coordinate when the app was revived
A list of points of the track we want to use for our data points.

Fortunately, I was able to obtain a list of coordinates for the missing data since I travelled the same path on the return journey (As can be seen on the map above).

The other 2 app state points of interest are reasonably easy to find - just find 2 data points that have a (reasonably) large time distance between them.

In order to process the data, I used a python library called gpxpy which provided some good utilities for reading and processing a GPX file.

With this library, I was able to find the crash point, the revival point, and the list of the points of the track. With this data, I interpolated the start/end times of the crash points onto the track data, and spliced it back into the dataset.

After exporting the data set, we achieve a map that looks like:

Quite clearly, this has a few limitations, for example, the calculated velocity through all of the data points is simply an average. However, this did provide me with an improved dataset which I could re-upload to Strava.

You can find all the source for this script on my github