N8n + urlwatch + telegram = Your personal jobfinder concierge

The other day a good friend of mine came to me saying:

Hey mimnix, I really have had enough of my current employment, low pay, and crazy hours, I need to find something else but I neither have the time to look for other opportunities because the job is destroying me.

It's a shitty not-okay situation obviously, I wanted to help but I wanted to do it my way. So, it was time to prepare a juicy Hackcipe.

Ingredients

• N8n: The workflow automation platform that doesn't box you in, that you never outgrow. Think about it as an Open Source IFTT-ish self-hosted platform, but better.
• Urlwatch: A command line tool for monitoring webpages for updates.
• A Telegram bot you own.

So, what we are going to create is an automation process that scrapes a job finder website (we will take Indeed as an example) using urlwatch, it extracts job application links, sends them to a N8n simple workflow exposing a webhook to clean up the input preparing it for a fresh telegram post, and shoots the output to your telegram bot...

... so that you will receive the job postings as soon as they're published.

Preparation

Step 1: Telegram Bot

Let's get started with a new telegram bot. You have to connect to @BotFather which is the official Telegram Bot-Maker, you can open it directly from this link: https://telegram.me/BotFather . And press or type /start when the chat with BotFather opens up.

Click /newbot, choose the name of your bot and continue following the instruction the bot will provide. At the end of the process, you will receive a message containing the API Token of the bot, copy it and save it securely, we will use it later.

Step 2: N8n workflow

Simple as it is, one workflow and three blocks: webhook, linked to the function block, linked to the telegram block. Here:

Set the webhook using:

• Authentication: none
• HTTP Method: POST
• Respond: immediately
• Response code: 200

Click on "production URL" and save it somewhere, we will use it later. Now, add the function block from the palette on the right side of the portal and paste the following snippet:

const output = []
var links = []

lines = items[0].json.body.text.split("\n")

for (line in lines) {
  if (lines[line].startsWith("+") && !lines[line].startsWith("++")) {
    links.push({
      json: {
        id: links.len,
        url: lines[line].replace("+","").trim()
      }
    })
  }
}

return links;

The code will filter the unnecessary payload that will come from urlwatch.

Step 3: Urlwatch

Let's setup Urlwatch. Its configuration consists of two files: urlswatch.yaml contains the application-specific configuration and urls.yaml that contains the urls we want to scan for updates over time and the filters to extract the information we want to extract, which in the specific case are job posting links.

urlsatch.yaml has the following content:

display:
  new: true
  error: false
  unchanged: false
report:
  text:
    footer: false
  telegram:  # WE WON'T USE THIS REPORTER
    bot_token: 'bot token' 
    chat_id: 'chat id' 
    enabled: false
  slack:   # WE WILL USE THIS REPORTER
    webhook_url: 'URL_TO_YOUR_N8N_WF_WEBHOOK'
    enabled: true

which basically means: send only new items, don't send errors through the reporting channel, don't send unchanged items. Don't add the footer payload to the reporter and enable the slack reporter (which actually is pointing to our N8n webhooks and urlwatch will never know that).

Profit. >_