N8N - Carousell.sg - Automatic Reply Bot
This has been superceded by Carousell GoBot project.
Context
I was looking for ideas to try out n8n.io's capabilities and picked this as the first big idea to try out.
Basically, I would like to n8n.io to handle chat messages and offers from Carousell, a Singaporean marketplace platform, similar to Facebook Marketplace.
Right now I would only like it to:
- Reply a standard response template on new messages and offers
- Forward to my personal Telegram bot to notify me via one additional channel
- Detect any low-balling (offering a price much lower than the listed price)
The problem was that Carousell does not have any public documentation on their APIs and I have to resort to inspecting the network traffic manually to achieve my goals.
Docker image: jarylc/n8n
Process
Automatic Triggers
There are 3 ways to this workflow is automatically triggered:
-> Webhook - Using Tasker on my Android device, it will send a GET request to this webhook everytime a notification from Carousell app arrives.
-> IMAP - Reads a dedicated inbox for e-mails from Carousell and trigger if so, acts as fallback for the above.
-> Cron - Every 30 minutes as fallback in case all of the above doesn't work. Note that there is a better to do this if standalone, you could just connect to the chat WebSocket and wait for messages.
Retrieve Chat Data and Preliminary Checks
When meddling around https://www.carousell.sg/inbox/received/ and inspecting the network trace, I found out that Carousell has an undocumented API to retreive that I could tap on.
This API call allows me to retrieve the list of messages and offers that I have recieved as a seller which I put into good use as the first step after the trigger.
It seems that only the Cookie header is required to authenticate myself.
The preliminary checks consists of checking if the chat list is empty, or the very last message contains my message signature.
Now this comes the hard part for this workflow, upon inspecting, Carousell uses SendBird for their chat platform which uses WebSockets instead of API calls.
I had to customize my n8n.io installation to include ws Node dependency and add it to NODE_FUNCTION_ALLOW_EXTERNAL environment variable to be used in the vm2 environment. This was made easy with my own Docker image on Docker hub using the environment variable ADDITIONAL_MODULES, jarylc/n8n.
After which, I had to mimic the WebSocket calls when chatting and came up with this final JavaScript node.
Custom function codes
Check & Split Checks
const fs = require("fs")
const seen = JSON.parse(fs.readFileSync('/data/carousell.json', 'utf8'))
const split = []
let save = {}
for (const item of $node["HTTP Request"].json["data"]["offers"]) {
if (item['state'] === 'A')
continue
let old_price = seen[item['id']] || 0.0
let latest_price = parseFloat(item['latest_price'].replace(',', ''))
let message_price_search = item['latest_price_message'].match(/^(\d{1,5}\.?\d{0,2})$|(\d+\.?\d{0,2}((?<=(\$|offer|quote|can|please|pls|quick|fast|sell).*)|(?=.*(\$|offer|quote|can|please|pls|quick|fast|bucks|ok|\?))))/gi)
if (message_price_search != null) {
let message_price = parseFloat(message_price_search[message_price_search.length - 1])
if (message_price > old_price && message_price > latest_price) {
latest_price = message_price
item['latest_price_formatted'] = (''+latest_price.toFixed(2)).replace(/\B(?=(\d{3})+(?!\d))/g, ",")
}
}
save[item['id']] = latest_price
if (item['id'] in seen && old_price >= latest_price)
continue
item['price_changed'] = old_price !== 0.0
item['low_balled'] = latest_price !== 0 && latest_price <= parseFloat(item['product']['price']) * 0.85
split.push({json: item})
}
if (split.length > 0) {
const fs = require("fs")
fs.writeFileSync('/data/carousell.json', JSON.stringify(save))
}
return split
Send Reply
const WebSocket = require('ws')
return await new Promise((resolve, reject) => {
let sent = false
const sendbird_subdomain = items[0].json['channel_url'].slice(0, items[0].json['channel_url'].indexOf('-carousell')).toLowerCase()
const client = new WebSocket('wss://ws-' + sendbird_subdomain + '.sendbird.com/?p=JS&pv=Mozilla%2F5.0%20(X11%3B%20Linux%20x86_64%3B%20rv%3A90.0)%20Gecko%2F20100101%20Firefox%2F90.0&sv=3.0.149&ai=F3CB6187-CB42-4CD1-95FC-1C46F8856006&user_id=344194&access_token=' + $node['Get Chat Token'].json['data']['token'] + '&active=1&SB-User-Agent=JS%2Fc3.0.149%2F%2F&Request-Sent-Timestamp=' + Date.now() + '&include_extra_data=premium_feature_list%2Cfile_upload_size_limit%2Capplication_attributes%2Cemoji_hash', {
perMessageDeflate: true
})
client.on('message', (data) => {
if (data.startsWith('LOGI') && !sent) {
sent = true
for (const i in items) {
try {
const item = items[i].json
let text = 'Hello @' + item['user']['username'] + '!\n\n' +
'Thank you for your '
if (item['latest_price_formatted'] !== '0') {
text += (item['price_changed'] ? 'new ' : '') + 'offer of ' + item['currency_symbol'] + item['latest_price_formatted'] + ' on'
} else {
text += 'interest in'
}
text += ' my item: ' + item['product']['title'] + '.'
if (!item['is_product_sold'] && item['product']['status'] !== 'R') {
if (item['low_balled'])
text += '\n\nWARNING: Offer price is more than 15% below listing price, it is too low and may not get a future reply unless you increase it!'
if (!item['price_changed']) {
text += '\n\nFAQ:\n' +
'» Where do I normally deal?\n' +
'Choa Chu Kang or Bukit Panjang area if I am not in office, Bencoolen area if I am.\n' +
'» What payment methods do I accept?\n' +
'In order of preference: Google Pay, PayLah, PayNow, Cash, CarouPay, Bank Transfer\n' +
'» What happens if I did not reply?\n' +
'Very likely you offered too low-ball of a price' + (item['low_balled'] ? ' (which you probably did)' : '') + '. If not, feel free to message me again.'
}
} else {
text += '\n\nHowever, this listing has already been ' + (item['is_product_sold'] ? 'sold' : 'reserved') + ' and not available anymore.'
}
text += '\n\n- @jarylc'
const msg = {
channel_url: '' + item['channel_url'],
message: text,
data: JSON.stringify({
offer_id: '' + item['id'],
source: 'web'
}),
mention_type: 'users',
mentioned_user_ids: [],
custom_type: 'MESSAGE',
req_id: Date.now()
}
client.send('MESG' + JSON.stringify(msg) + '\n')
} catch
(err) {
reject(err)
}
}
} else if (data.startsWith('READ')) {
client.terminate()
resolve(true)
}
})
client.on('error', (err) => {
reject(err)
})
client.on('open', () => {
setTimeout(() => {
client.terminate()
resolve(true)
}, 10000)
})
}).then(_ => {
return []
}).catch(err => {
throw err
})
Telegram Notify Flow
The other branch just forwards this to my personal Telegram bot so that I can be notified via another channel. Right now it will also tell me if the offer was a low-ball or not.