🔥 Hacking Google CSE to find niche candidates

In today's edition, I'll explain what a Custom Search Engine is, how to create your own, and demonstrate how it can be used to find what (or who) you are looking for!

Sep 20, 2021

I’m going to demonstrate today how you can make a rookie recruiter in your team source as efficiently as an experienced recruiter.

🤯 Crazy right? Stay with me and I’ll walk you through all the steps to leverage Google’s Custom Search Engine to achieve this.

Do you X-ray the same sites over and over? Do you ever get tired of writing a search string with the same search operators in it?

There’s an easy way to eliminate that step by creating a Custom Search Engine in Google. CSE is a great tool to automate your favorite search strings. It’s a simple solution for you to save time with searches and make your job more productive.

CSE also prevents Google from displaying Captchas so you are not going to be blocked by Google every time you are using complex strings. Pretty dope right?

It’s like building your own mini Google to search only the sites and keywords that you want to search and eliminate specific sites or keywords that you don’t want.

While it may sound complicated, it’s not. You don’t have to be a developer or even extremely technical to create one.

Let’s build a simple one to X-Ray LinkedIn from scratch 🎉

Step 1 — Build your CSE

To begin, go to 👉 CSE dashboard page

In the Sites to search section, add the pages you want to include in your search engine. You can include any sites you want, not just sites you own.

Since we are creating a CSE to X-Ray LinkedIn, let’s add www.linkedin.com/pub* and www.linkedin.com/in* 👇

The * at the end of each URL is part of the Google URL formatting and it’s needed to include all pages that begin with that URL.

For example, the pattern 'www.linkedin.com/pub' will only match the single page 'www.linkedin.com/pub'. To cover all the pages where the URL starts with ' www.linkedin.com/pub', you must explicitly add a '*' at the end.

Click Create.

Now you have an option to “Get code” and add your new Custom Search Engine on your website or chose “Public URL” and test it first 👇

There is no need to add the CSE on your website and don’t worry, even if you chose the “Public URL” you can always go back and get the code for your website.

Step 2 — Simplified search

Within a Custom Search Engine, your string can be much shorter and cleaner as you don’t have to include site operators.

Instead of a string that looks like this 👇

"software developer" OR "software engineer" site:www.linkedin.com (inurl:com/pub | inurl:com/in) -inurl:pub/dir

You can search in your CSR with a much simpler string like this 👇

"software developer" OR "software engineer"

Why this search string?

LinkedIn uses two directories (pub and in) for public profiles. Because of this, we’ll need to use the inurl elements (com/pub & com/in)to tell Google that we only want to see profiles in our search results.

Additionally, LinkedIn placed their “directory” listings (dir) in the directory immediately following the (pub) directory so we have to tell Google to exclude that with (-inurl:pub/dir) or we take the chance of getting results that are not profiles but actually a page containing a list of them. You can read more about this here.

Step 3 — Add refinements

Google CSE in the free version has a limitation of your results set to 100. One easy way how to overcome is to create Refinements.

Refinements will appear as labels in your CSE and there is no limitation to how many you can add. Also adding refinements will allow you to exceed the 100 limit.

If you’d like to add refinements, just click on the Search features tab on the left side from the control panel of your CSE and click the Add button👇

Let’s add a refinement called With Email and add keywords to search for LinkedIn profiles of candidates with e-mail addresses on their profile page.

Keywords: @ OR email OR "e-mail" OR "send me an email" OR "send me an e-mail" OR "e-mail me" OR "email me" OR "at * com" OR "email me at" OR "e-mail me at"

The last step is to assign this refinement to the sites you created. To do this, you just go back by clicking on the Setup tab on the left side from the control panel and scroll down to find the Sites to search section.

Check all the sites that you would like to be connected with this label, select the label from the dropdown, and hit the Apply button.

Once the label is connected to the sites you screen should look like this 👇

When you add the new Label you’ll see it in your Google Custom Search Engine as a new tab 👇

Another example is using keywords to search for active candidates on LinkedIn. Add a new refinement called Active candidates and use these keywords to create a new label to only see candidates who are actively looking to jump ship.

(seeking OR seeker OR "looking for" OR "in search of" OR "open to" OR "new job" OR "actively pursuing" OR "pursuing new" OR "searching for" OR "new opportunity" OR "new opportunities")

This is another example to search for Linkedin profiles where candidates publicly put down their Gmail address 👇

The With Gmail in CSE now will only show LinkedIn profiles with a Gmail address.

The possibilities are endless. Now you know that creating CSEs is not rocket science! Creating CSEs for your own and your peers’ use has numerous advantages.

You can pass this on to a fresher in your team and they can be as efficient and productive as a senior recruiter without understanding any of the intricacies and nuances of boolean search and query strings.

Here’s the link to the final CSE that we created.

Bonus — How to find public CSEs from expert recruiters

To find additional CSEs, you can google any of the following (replace LinkedIn with a topical keyword related to the type of CSE you seek). Here are a few tips as recommended by Glenn in his post on the SourceCon blog:

This is Dean Da Costa’s recommended method (focusing on links that are meant to be public CSEs):
domain:cse.google.com/cse/publicurl LinkedIn
This gets you many more results, but might also add additional false positives to filter through:
(domain:cse.google.com/cse/publicurl OR “cse.google.com/cse?cx”) LinkedIn
If you are using more narrow keywords than LinkedIn (e.g., financial example below) and you don’t get enough results, you can loosen it up a bit as follows:
“sovereign wealth fund” (domain:cse.google.com/cse/publicurl OR “cse.google.com/cse?cx” OR “google cse” OR “google custom”)
Do you want the mega-list? This vendor claims to have compiled a list of almost every web domain that uses Google CSEs (over 552,000 as of this writing!). Unfortunately, it only displays the first 50 sites for free.

Also, check out this post from Irina where she has curated 17 Google Custom Search Engines for Sourcing 👇

Here’s another list of 10 custom search engines for recruiters.

I figured that there's value in my approach, and so I decided to share it with you today. If you enjoyed this post, do share your thoughts with me and consider sharing it with friends, or subscribing if you aren’t already.

That’s it for this week! Hit me up if you have any stories, feedback, or insights to share. Otherwise, see you next week!

I'm here if you need help or have questions,

Raj Abishek, Co-Founder @NurtureBox

10x Recruiter