The Roycoding Blog

The shortest definition of embeddings?

Sat, 19 Nov 2022 00:00:00 +0000

The shortest definition of embeddings?

19 Nov 2022 - This is a post on my blog. Get the RSS feed!

Embeddings are a very important concept in deep learning and several related techniques. They are powerful and flexible and at the core of many modern machine learning systems. But, they are also tricky to describe.

Here is my attempt to create a very succinct definition and description of embeddings.

Embeddings are learned transformations to make data more useful

What does this mean?

Learned means that the parameters of the transformation are determined using the data, rather than just a fixed algorithm. The parameters are optimized for some task using the data as input. E.g. teaching a small neural network to determine if words found in a text dataset are from the same sentence. The network needs to transform the input text data in a way that makes this task as easy as possible (this is basically word2vec).
Transformations are how data is changed from being represented in one way into another. This typically means mapping data from one vector space to another, such as taking words from a vocabulary of 10k words that are one-hot-encoded in a 10k dimensional space to a 100 dimensional "dense" vector space.
Data can be any type of information: words, sentences, images, audio, webpages, time series. Often the input data is considered a "feature" for machine learning.
Useful means that the new representation of the data contains more information in a way that makes the ultimate task easier. Useful can mean:
- Grouping similar items together or warping the representations to make existing groups more easily identifiable (e.g. linearly separable).
- Representing the data in fewer dimensions, whether to more easily encode relationships between the data or for computational efficiency.
- Being reusable for many closely related tasks.

Embeddings are incredibly useful for many data related tasks. They are especially useful for things like search (e.g. what other images are like this one?) and recommendations. More generally they are useful for working with data that is not inherently numeric, such as text, images, or audio.

While I didn't touch on the technical aspects of creating or using embeddings, hopefully this is helpful as a conceptual overview for you.

Note: My definition is slightly different from others, because I don't think that embeddings strictly need to be in a lower dimensional space, but that is often the best choice for the task at hand. What exactly qualifies as an embedding can get a little fuzzy when considering basic, non-learned encodings (does one-hot encoding count) or learned transformations that are only used as internal (latent) representations, etc.

Why I don’t screen resumes

Sat, 02 Oct 2021 00:00:00 +0000

Why I don't screen resumes

Roy Keyes

2 Oct 2021 - This is a post on my blog. Get the RSS feed!

Probably the most controversial recommendation I make about hiring for data roles is to ignore resumes. In this post I'll talk about what that really means and why (and when) I recommend that.

Too many damn resumes

Back around 2017 something changed. I was hiring data scientists, as I had done before, but there was one big difference: simply too many applicants! I was (again) working at a young startup with little to no name recognition, but applications were flooding in. The volume of candidates had grown incredibly in just a couple years. My previous process could not handle this new scale. In order to handle this increased volume, I implemented a resume scoring process designed to be fast and as fair as possible.

I soon found that even this was not efficient enough to handle the growing volume. I decided to take the more radical step of ignoring resumes completely. And it seemed to work.

Why ignore resumes?

Yes I'm suggesting that you ignore the carefully crafted career summary that your job hopefuls are sending in. Despite the hours (and even money) that they've poured into creating those resumes[1], one thing can make them irrelevant: applicant volume.

But let's talk about what this actually means. What do I mean by excessive applicant volume? If you are in a situation where your applicant numbers greatly outpace your hiring resources, resume screening is likely to be a bottleneck. For me that meant hundreds or thousands of applicants, where I was the only one who could dedicate time (and expertise) to evaluating resumes.

Sure, you can have people glance at resumes or use keyword filtering (or worse, some sort of "AI" resume filter), but this is not going to give you the effective and fair screening that you want. Resumes themselves are often messy and also allow you to over-index on certain things, such as schools, past employer names, and even more insidious signals to bias against.

So what do you do, if you don't look at resumes?

Resumes are nice, if quite imperfect, summaries and advertisements of a candidates skills and experience. The whole point is that a short document can convey their fit for the roles they're applying for. So what's a reasonable way to not actually look at resumes?

If you've read my book or other blog posts, you'll know that the first steps in hiring are to understand your needs, identify the tasks and skills needed, and the skills you want to evaluate. In a situation with a very high candidate volume, I recommend using a technical skills assessment as a first step screen. Typically I'll use a relatively short, automated, online screen that covers basic skills needed for the role[2].

I give this assessment to almost all applicants, sometimes (cheating and ) briefly checking to see if the candidate was in the wrong place (e.g. machinists applying for ML roles, which I've seen before!). This assessment covers skills that would be otherwise probed in the interview process, but in this case are just moved to the first step.

The whole point of this is to reduce the size of your candidate funnel. That tends to happen in a couple ways. Some people simply won't take a skills assessment, because either they refuse to out of principle (probably a bad idea in this industry) or because they see the assessment as too much friction for this specific role [3]. The other way is of course candidates not doing well enough on the assessment, relative to your perceived needs or simply the rest of the candidate pool.

OK, but what about the resumes?

The skills assessment only covers a portion of the total evaluation of a candidate. You're still going to interview them by talking to them and going more in depth on their background and technical skills.

For this process, I actually do still use people's resumes, but it's more of an additional piece of information, rather than a something to screen against. Imagine two otherwise identical candidates, the one with more time spent in your domain or using the techniques you need, as reflected in their resume, is probably the one you'd choose.

So the headline is somewhat misleading, but flipping this process has proven useful if dealing with the candidate volumes that I've seen, especially when hiring for early career data roles in particular[4].

Hiring strategy and process are incredibly difficult and controversial things. My main advice is to put more effort into it than you think you might need to, create a plan, and be ready to change as needed. In this case, I ended up making a somewhat radical change, but this was in response to the seemingly exponential change happening in the applicant pool. It seemed to work.

For more on this topic, please check out my book, subscribe to my blog (--> the RSS feed), and follow me on Twitter, @roycoding.

[1] And cover letters

[2] I plan to cover this in another blog post, otherwise, please check out the chapter in my book.

[3] I do my best to accommodate candidates that have logistical difficulties with the assessment. You want your candidates to have as positive an experience as possible, so flexibility is always good.

[4] In situations where you have a lower ratio of applicants to resources (e.g. more senior level roles or working with a large HR/recruiting team), looking at resumes is totally reasonable. On the other hand, having used the "no resumes" process, questions about how fair it is to use resumes to make initial decisions seem more tangible.

How I wrote my book – the technical details

Wed, 18 Aug 2021 00:00:00 +0000

How I wrote my book - the technical details

Roy Keyes

18 Aug 2021 - This is a post on my blog. Get the RSS feed!

I recently released a book called Hiring Data Scientists and Machine Learning Engineers. I self-published the book via a platform called Leanpub. This post is about how I actually did my writing, primarily the technical details of my setup and workflow.

Let's say you've decided to self-publish a book...

When I decided to actually write a book, my main criteria was being able to create and distribute something high quality with the least amount of effort, while retaining a lot of control over the final product. As mentioned in my early blog post that talked about my motivation for writing the book, I came across Leanpub because a friend had published a book there.

Why Leanpub in particular?

In the era of social media, blogs, podcasts, and streaming video creators can essentially deliver directly to their audiences. Similarly, self-publishing is not only easier than ever, but also just makes sense for a lot of people. This is the age of fast turnaround and disintermediation, which is just a fancy term for removing the middle man (and usually replacing them with a "platform").

But why Leanpub? There are a lot of choices for self-publishing a book these days. I chose Leanpub for a few specific reasons:

It's aimed primarily at technical content (computery stuff).
It already has a lot of good data science and machine learning-related books and courses.
It provides several nice authoring workflows (see below).
It pays a good royalty rate (80% to the author).
It allows you to publish and sell in-progress works.
It allows you to set a minimum price and allows readers to pay what they want over that.
It allows you to create coupons (including programmatically via API).

Overall I like how pro-author they are.

How I wrote my book

Leanpub has several authoring workflows available, including fully in-browser authoring, but I decided to use a their more developer-style workflow. It starts with writing the content locally in Markua, Leanpub's book oriented dialect of Markdown, using git for version control, pushing changes to a Github repo, and then finally Leanpub pulling the latest version of the source and building the ebook files in several formats for you.

I really liked writing that way, because it's similar to how I already write notes, documentation, and blog posts in my text editor (VS Code these days). Markua is close enough to vanilla Markdown that the Markdown extension I use with VS Code renders it well enough for writing. In the end you need to have Leanpub build updated ebook files to know for sure how things will look, but this is definitely not a deal breaker.

Pretending to be a designer

I enjoy making graphics, and even though I am a certified amateur, I decided to try my hand at creating the cover for my book. I created the cover and other graphics using Inkscape, which is a pretty incredible open source vector graphics editor.

The graphics I used inside the book, which were primarily decorative, were a mix of vector graphics I created by hand, modified public domain vector graphics, and equations typeset using LaTeX within Inkscape.

The build process Leanpub uses allows you to use SVG files directly, so instead of exporting my graphics to a bitmap format I used SVG, with the exception of the cover graphic.

Writing on a 6" screen

At some point while writing my book, it occurred to me that I could probably write directly on my phone. This is probably viewed as masochistic by many, but it seemed like a fun thing to try "because it was there".

To get this to work I installed an app called Termux, which is a terminal emulator and Linux environment for Android, available via F-Droid, an Android app store with only open source apps. Termux allows you to install and run a lot of the programs that Linux/Unix users expect to be able to run from the command line, including git.

Next up I installed git and ssh via the Termux package manager:

pkg install git

pkg install openssh

Once I installed git, I set up a new Github key on my phone following the usual process, pulled down my book repo and was ready to start writing. Since I was writing in a Markdown dialect, I looked for a Markdown editing app, ultimately settling on an app called Markor. The workflow is then identical to writing on a full-sized computer.

Write some.
Commit the changes in git.
Push the changes to Github.

As you might imagine, long-form writing on a virtual phone keyboard is not the funnest way to write. The nice thing about a phone is that they tend to handle voice typing well. While I did that some, I prefer to type when writing. Fortunately the keyboard that I normally use is very small and designed with portability in mind: the Keyboardio Atreus. The Atreus is a small, split, columnar layout, mechanical keyboard. Yes, you need to be a keyboard geek to use the Atreus, but look, you've read this far for a reason.

So aside from small edits, I did most writing on my phone with the same keyboard I used the rest of the time on my computer.

What might be improved

Overall I've been happy with the choices I made and the process, but it was certainly not without room for improvement.

It would be nice to be able to build the book completely locally, if only for the sake of portability. What if I decide I want to take my Markua source elsewhere? Would I need to convert to a completely different format?

Another issue was sharing my work with others for editing and commentary. For people that are used to a Github centric workflow, I could share my source text with them and they could comment or make proposed changes just like they would with code. But for other people who helped me with the book, but who were not used to working with git, I would send them PDF or raw text, which was not ideal. It was not a showstopper, but it could certainly be improved.

What does it all mean?

Writing a book can be a long, often grueling process. It was nice to be able to use the tools I'm used to and to branch off into the (practical[?], but) whimsical as well.

If you have any questions about this process, please email/message me.

And don't forget to check out my hiring book at dshiring.com.

What’s hard about hiring data roles

Mon, 09 Aug 2021 00:00:00 +0000

What's hard about hiring data roles

Roy Keyes

9 Aug 2021 - This is a post on my blog. Get the RSS feed!

Hiring is hard and data hiring has special challenges

I recently released a book about hiring called Hiring Data Scientists and Machine Learning Engineers. Some of the questions I've fielded related to my book are:

Why is hiring data roles hard?
What makes DS and MLE hiring specifically difficult?

These questions are at the core of why I wrote my book. Hiring in general is difficult, but hiring for data roles comes with some special challenges. In this post I will lay out my view on why data roles are specifically hard to hire for.

Hype and hope

Data science and machine learning have emerged in the last ten years as particularly promising approaches to a number of business problems. Organizations have more access to more data than ever and the hope that business value can be extracted from that data has reached a fever pitch.

The hype around data science and machine learning (and something called "artificial intelligence") has driven all sorts of organizations to build out data teams. The hype has also led a huge number of people to enter the field in the hopes of landing an interesting and highly compensated career (remember the "sexiest job of the 21st century"?).

This is still the early days of this era of data science and machine learning. While there has been some evolution of roles and titles, the field is extremely broad and there is still a lot of disagreement on what exactly data scientists, machine learning engineers, and related roles actually do. A data scientist at one company may have a very different skill set and set of duties than a data scientist at another company, despite the shared title.

Hiring amid the hype

The newness, hype, and lack of clarity create a number of challenges for people trying to hire for these roles.

The fact that these fields are relatively new means that there are few really experienced data scientists and machine learning engineers, but also there are few people that are really experienced in hiring for these roles. The newness means that the easiest thing to do has been to simply use or adapt hiring strategies from more established roles, such as for software engineering. This creates challenges, as the wrong assessment strategies will lead to more false positives and false negatives in the process.

The newness also means that finding those experienced candidates can be very difficult and ultimately very expensive when you do find them, since the supply is so low. Organizations are faced with the fact that experienced talent may simply be outside their budgets.

The wide breadth of topics, skills, and technologies that fall under data science and machine learning means that there is often a mismatch between the open role and the candidates. It's often not clear to either side what the other brings to the hiring table, and this can often lead to confusion and frustration.

"I thought this was a data science role!"

"It is!"

This lack of alignment and understanding on the basic role definitions is a common source of issues.

The hype has led to a huge influx of people joining the field, leading to a glut of early career people applying for DS and MLE roles. This means that a hiring manager looking to fill more junior level roles is likely to be overwhelmed with the volume of applicants. A team looking to fill both junior level and senior level positions may be faced with two extremely different sourcing problems: too many and too few applicants at the same time.

Overcoming these challenges

To hire effectively and efficiently, you need to understand these challenges and design a hiring strategy to address them. Core aspects of such a strategy are:

Understanding your business needs and determining the specific roles that match.
Creating a job ad that crisply describes the role you are hiring for.
Understanding your resources, your constraints, and the market overall.
Designing a recruiting, assessment, and interviewing process that's optimized with your resources and constraints in mind.

You can read more in depth on these issues in my book, which is available at dshiring.com.

I wrote a book

Tue, 13 Jul 2021 00:00:00 +0000

I wrote a book

Roy Keyes

14 July 2021 - This is a post on my blog. Get the RSS feed!

I never really set out to write a book.

I wrote a book. A book about hiring. You can buy it at dshiring.com.

Back in 2020 we had a global pandemic. Like many people, I was laid off (along with most of my awesome team). Like some people, I decided to try to do something a little different during that time. Somehow I ended up writing a book. This blog post is about that book and how it came into existence.

When not working in a full-time role (or infrequently, in addition to) I usually take on consulting projects. Earlier in my career those projects where mostly about helping companies implement data science projects, but more recently the projects I've worked on have centered more on strategic decisions related to data-related product and project planning, and more and more often about team planning and hiring strategy.

In parallel with consulting, last summer I also decided to see if I could land on an idea for creating my own software business. Given my experience and interest, one of the more obvious ideas was building software to improve the process for companies hiring data scientists and machine learning practitioners.

Hiring for data roles can be difficult. One of the main sources of difficulty is often the shear scale of the problem. Especially when hiring for more junior roles, the number of applicants you need to process can be overwhelming. I was looking for an idea to build something that would help improve that common situation.

While investigating business ideas and also talking to clients, some of the same themes kept coming up over and over:

How do you know which roles to hire?
How to you find the right people?
How can you tell who knows what they're doing?
How do you deal with the large volumes of candidates?

On the one hand, people were willing to pay me to answer these questions, but on the other hand, I was hearing these same questions enough, that it made sense to write them down.

So I decided to write some blog posts.

What about the book?

My initial plan was to write some blog posts that I could then send to people who asked me questions about hiring. At the same time, I was struggling to find an product idea that really felt like it would make sense. Two things happened. The first was I came across some advice that the first "product" of your software business need not be actual software, but could be a newsletter, or something else that would help you grow mindshare, while you were working on your software. The other thing was that I saw Joel Grus release his latest book Ten Essays on FizzBuzz. Unlike his last book, Data Science from Scratch in Python, he decided to self publish this book on a platform called Leanpub.

Naturally I thought, "oh, maybe I should just write a book instead of boring old blog posts. How long could it possibly take?". The reader will of course know that that was one of those fantasies that your brain likes to tell itself sometimes. In reality, instead of a couple months, the book took me closer to eight months to fully complete.

Self publishing in the internet age

When my dad was not much older than I am now, he decided to write a very niche book and self publish it. Needless to say, at that time things were much different. I remember stacks of boxes of his book sitting in the garage for years, as he slowly sold them, eventually giving many copies away to win back valuable space[1].

Today the world works very differently. My main reason to self-publish was because I assumed (I still think correctly) that it would be much quicker and take less effort on my part. Depending on your goals, self publishing today is probably the best choice for most people who have the inclination to write a book. The barrier to entry is lower than it has ever been, just was with most media publishing (video, audio, etc).

I ended up publishing my book via Leanpub, a publishing platform geared primarily towards computing related technical books. Leanpub has very author friendly terms (e.g. 80% of the purchase price goes to the author) and also is geared towards people like myself, who would prefer to write in something simple like Markdown[2]. I'm planning another post detailing the specific workflow I used to write my book.

Self publishing is not without its drawbacks. The royalty structure from a publishing platform like Leanpub is much better than with a traditional publisher, but marketing the book falls almost entirely on the author's shoulders. Somewhere an economist is doing a calculation to find how these scenarios compare, but it's certainly not clear to me how the ultimate expected compensation compares.

In the end you probably shouldn't write a book with the expectation of making a lot of money. I suspect that, overall, I will come out better monetarily than my dad's effort did, though (sorry, Dad!).

I'm now a published author

Well, sort of. Self published, anyway. But that's fine and my book probably wouldn't exist if I'd decided to try to publish with a traditional publisher, as that sounds like an order of magnitude more effort[3]. I'm not sure I would have gone through with it if I had decided that that was the only "correct" way to do it.

I'm glad I wrote this book, as I ended up having a lot of great conversations while making this, including the interviews that I included in the book. It's also been great to hear feedback (and even some appreciation) for the book. I hope I've provided readers with some useful insight and ways to think about the process of hiring data science and machine learning people.

If you haven't checked my book out yet, please take a look at dshiring.com. And follow me on Twitter @roycoding!

Footnotes

On occasion a listing for his long out-of-print book will come up online with an absurdly inflated price due to unrealistic pricing algorithms.
Leanpub uses it's own version of Markdown called Markua, which is designed for writing books.
If you're nonetheless interested in offering me a publishing contract, my emails are open.

Notifications with Keybase: You get a puppy!

Wed, 22 Jul 2020 00:00:00 +0000

It's 2020 and I made an RSS feed for my blog (with Python)

Thu, 21 May 2020 00:00:00 +0000

Multi-armed banditry in Python with slots

Mon, 22 Aug 2016 00:00:00 +0000

A year of working remotely

Mon, 29 Feb 2016 00:00:00 +0000

4 Core attributes of a data scientist

Sun, 17 Jan 2016 00:00:00 +0000

Computer generated art (in Python)

Sun, 21 Sep 2014 00:00:00 +0000

MapReduce Patterns (in Python)

Wed, 17 Sep 2014 00:00:00 +0000

Klackers strategy: Monte Carlo FTW

Mon, 19 May 2014 00:00:00 +0000

Github Gists: Blogging for the lazy

Tue, 11 Mar 2014 00:00:00 +0000