Hello,

I know it has been a while since the last issue, and in case you are clueless, this is a newsletter where I share a few interesting reads that I find online. Most of them are usually technical but there can be some variations from time to time. Hope that is something that you are still interested in.

Here are two articles that I really enjoyed reading through last week.

How to build a TCP proxy (4-part series)

I work at Requestly which is an easy-to-use proxy for frontend developers. Think of it like burp but for QA engineers rather than bug hunters. So I am always eager to learn more about proxies.

Tools like Burp, Requestly, Charles Proxy, etc. all act as HTTP proxies (layer 7 proxies). This is usually enough for most use cases because most of the web that we use day to day is built on top of HTTP.

To intercept your HTTP traffic with any of these proxies, you need to add the proxy’s IP and port to your system/app’s proxy settings. Even if you are integrating it inside your source code, most HTTP clients like Axios (for NodeJS) or RetroFit (for android) accept a proxy attribute to alter this setting across your app with just a few lines of code (Basically, you are changing the destination for all your requests)

In the case of a TCP (layer 4) proxy, you need to find a different way to redirect your requests to the proxy.

This article goes through all the details of building a generic TCP proxy and provides an interesting solution for forwarding all your TCP traffic to a proxy.

The author does so by creating a simple fake DNS server, running it along with the proxy and changing the default Domain server that his mobile uses to resolve domain names (which seems surprisingly easy to do).

We know that every request on the web first goes to a DNS server so that we can resolve easily recognisable URLs like “nsrcodes.com” to their IP address (which is what your browser needs to make the request).

Now, if you can control which IPs the domain names map to, you have essentially got control of redirecting any request to wherever you wish to send it, which in our case is our PROXY server.

So, all the requests first go to this fake DNS server and this server decides whether it wants to send these requests to the proxy or hit an actual DNS server and return a response. Genius!

Along with the implementation for this, the article also contains simple-to-follow explanations on how to build a TCP proxy from scratch.

Read the article

Redis Explained

For those unfamiliar with Redis, it is a general-purpose database that stores data directly inside the main memory

I recently had the chance of using Redis inside our infrastructure. We used it as a simple time-based cache for frequently accessed data. Since then I have been eager to know what else Redis is capable of and more importantly, how it actually works “in-memory”

The problem with an in-memory database is how you handle the persistence of data. Since the data is all in memory, shutting the system down (either because of faults; which are bound to happen; or just regular redeployment) will lead to loss of data. Redis handles this quite well with an async process that runs periodically to save snapshots or append-only files (file with a log of all write operations on the database for easy recreation of the current state).

My favourite part is at the end where it describes how Redis (which is inherently single-threaded) spawns forks of its main process to run this async persistence workflow (I had no clue how these two processes would share the addresses of the complete memory, and not cause conflicts)

Another problem is the hardware limitation. Since this is an in-memory database, you can only have so much ram. The obvious answer is to shard the data across several nodes, but since I had never actually worked with databases that much, I never understood how sharding works. This article explains how Redis handles sharding when deployed as a cluster of Redis nodes

Read The article

Hope this was useful, let me know your thoughts on anything that you find interesting from the above two articles. Until then…

Although the task remains equally challenging throughout, it is towards the end of it that I realize that I have to make sure that this does not break everything. I might get better at recognizing this at the start but for now, this is a habit I need to fix.

But this has also helped me sometimes because I have learned that bugs aren’t always an overlooked edge case or an embarrassing typo. Sometimes they are there to highlight specifically what I do not know about. And I don’t mean that in the context of a codebase. These special bugs tell me what more I have to learn in the domain I have been working in.

Here’s an example…

We recently had a user tell us that our most basic feature is giving an unusual bug. Our whole team had not seen this bug before. This was a special bug that I mentioned earlier and here is my debugging story.

Do you have any such bugs that you are glad you encountered?

Leave a comment

I generally have a plan about such drastic descisions but this time I don’t. I just know that I always wanted to have a blog (my corner on this beautiful internet) and a channel for openly expressing my thoughts. I have embarrasingly not done anything to move forward in that direction and I plan to change that this year.

All this starts with the blog being updated and me personally making a few changes in my routine to incorporate a writing schedule. I don’t expect it to be easy, but I do expect to stick with it this time.

So to the one other person reading this, thanks for sticking around. I will try not to disappoint!

Subscribe now