This is an article to show how to setup a scheduled function that checks and restores a missing webhook as well as the snippets when receiving the webhooks and storing them.
Why?
We found that Shopify would intermittently remove our webhooks. This would make the entire synchronization of the system go out of whack and some records would slip through the net while every day, the process to tidy these all up would leave the data more and more unreliable.
So Shopify want your webhooks to respond within 1 second. This is about enough time to store the record but that's it. Processing the record must be done at a later time or in a different process but sometimes it simply will not process it all and respond to Shopify within 1 second.
How?
So let's begin with the code snippets as these might answer your question to begin with and then we'll do the easy bit on how you set this up in the CRM. I'm using ZohoCRM here despite storing the record in ZohoCreator as the app and data processing is mostly done in ZohoCreator. I could have setup webhooks to go directly to Zoho Creator but I don't find Zoho Creator reliable when receiving and responding to webhooks.
Important!: For the REST API to detect what webhooks exist, it can only detect what webhooks it initialized. If you added webhooks via another app or via the Shopify Admin interface, then the following code won't be able to check and restore them.
The Code Snippets: The Shopify Order Update notification
First create a CRM function called Shopify - Webhook - Order Notification. It's a standalone function but you'll need to hover the mouse over it after creating the function so that you can enable the REST API endpoint of it. This will need to be triggered whenever there is an update to the order (especially when tracking details are changed on the order).
/* ******************************************************************************* Function: fn_Shopify_Webhook_OrderUpdate Label: Shopify - Webhook - Order Update Trigger: Function executed when a webhook from Shopify is sent to ZohoCRM (Order Update) Inputs: String crmAPIRequest Outputs: String crmAPIResponse Date Created: 2022-01-17 (Joel Lipman) - Initial release - Forwards webhook to Zoho Creator Date Modified: 2024-04-22 (Joel Lipman) - Minimized amount of processing to simply storing the record in Zoho - Run HMAC SHA 256 verification ******************************************************************************* */ // // need to respond in less than 1 second m_Payload = crmAPIRequest.toMap(); m_Blank = Map(); // // you created an app that generated a client ID and a client secret and resulted in an access token // the client secret used when generating the access token should be entered here v_ClientSecret = "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"; // // store the record in ZohoCreator // I have a custom form in ZC with the fields "Event_Type" (single-line) and "JSON_Payload" (multi-line) m_CreateRecord = Map(); m_CreateRecord.put("Event_Type","Order Update"); m_CreateRecord.put("JSON_Payload",m_Payload.toString()); r_CreateRecord = zoho.creator.createRecord("myOwnerName","myAppName","myFormName",m_CreateRecord,m_Blank,"myCreatorConnection"); // // build the hash based on the payload body v_Data = ifnull(m_Payload.get("body"),""); v_VerifyHash = zoho.encryption.hmacsha256(v_ClientSecret,v_Data,"base64"); // // retrieve Shopify's Hmac SHA256 from the header v_ShopifyHash = ""; if(m_Payload.get("headers") != null) { if(m_Payload.get("headers").get("x-shopify-hmac-sha256") != null) { v_ShopifyHash = m_Payload.get("headers").get("x-shopify-hmac-sha256"); } } // // build response to Shopify m_ResponseHeader = Map(); m_ResponseHeader.put("status_code",401); // // compare our hash vs the received hash if(v_VerifyHash == v_ShopifyHash) { m_ResponseHeader.put("status_code",200); } // // respond return {"crmAPIResponse":m_ResponseHeader};
- /* *******************************************************************************
- Function: fn_Shopify_Webhook_OrderUpdate
- Label: Shopify - Webhook - Order Update
- Trigger: Function executed when a webhook from Shopify is sent to ZohoCRM (Order Update)
- Inputs: string crmAPIRequest
- Outputs: string crmAPIResponse
- Date Created: 2022-01-17 (Joel Lipman)
- - Initial release
- - Forwards webhook to Zoho Creator
- Date Modified: 2024-04-22 (Joel Lipman)
- - Minimized amount of processing to simply storing the record in Zoho
- - Run HMAC SHA 256 verification
- ******************************************************************************* */
- //
- // need to respond in less than 1 second
- m_Payload = crmAPIRequest.toMap();
- m_Blank = Map();
- //
- // you created an app that generated a client ID and a client secret and resulted in an access token
- // the client secret used when generating the access token should be entered here
- v_ClientSecret = "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";
- //
- // store the record in ZohoCreator
- // I have a custom form in ZC with the fields "Event_Type" (single-line) and "JSON_Payload" (multi-line)
- m_CreateRecord = Map();
- m_CreateRecord.put("Event_Type","Order Update");
- m_CreateRecord.put("JSON_Payload",m_Payload.toString());
- r_CreateRecord = zoho.creator.createRecord("myOwnerName","myAppName","myFormName",m_CreateRecord,m_Blank,"myCreatorConnection");
- //
- // build the hash based on the payload body
- v_Data = ifnull(m_Payload.get("body"),"");
- v_VerifyHash = zoho.encryption.hmacsha256(v_ClientSecret,v_Data,"base64");
- //
- // retrieve Shopify's Hmac SHA256 from the header
- v_ShopifyHash = "";
- if(m_Payload.get("headers") != null)
- {
- if(m_Payload.get("headers").get("x-shopify-hmac-sha256") != null)
- {
- v_ShopifyHash = m_Payload.get("headers").get("x-shopify-hmac-sha256");
- }
- }
- //
- // build response to Shopify
- m_ResponseHeader = Map();
- m_ResponseHeader.put("status_code",401);
- //
- // compare our hash vs the received hash
- if(v_VerifyHash == v_ShopifyHash)
- {
- m_ResponseHeader.put("status_code",200);
- }
- //
- // respond
- return {"crmAPIResponse":m_ResponseHeader};
The Code Snippets: The Shopify Inventory Level Update notification
Note that this one is not the "Inventory Item Update" as I get the two muddled up very often and they don't do the same thing. We want this triggered whenever the stock level of a product changes.
/* *******************************************************************************
Function: fn_Shopify_Webhook_InventoryLevelUpdate(String crmAPIRequest)
Label: Shopify - Webhook - Inventory Level Update
Trigger: Function executed when a webhook from Shopify is sent to ZohoCRM (Inventory Update)
Inputs: String crmAPIRequest
Outputs: String crmAPIResponse
******************************************************************************* */
//
// need to respond in less than 1 second
m_Payload = crmAPIRequest.toMap();
m_Blank = Map();
//
// you created an app that generated a client ID and a client secret and resulted in an access token
// the client secret used when generating the access token should be entered here
v_ClientSecret = "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";
//
// store the record in ZohoCreator
// I have a custom form in ZC with the fields "Event_Type" (single-line) and "JSON_Payload" (multi-line)
m_CreateRecord = Map();
m_CreateRecord.put("Event_Type","Order Update");
m_CreateRecord.put("JSON_Payload",m_Payload.toString());
r_CreateRecord = zoho.creator.createRecord("myOwnerName","myAppName","myFormName",m_CreateRecord,m_Blank,"myCreatorConnection");
//
v_Data = ifnull(m_Payload.get("body"),"");
v_VerifyHash = zoho.encryption.hmacsha256(v_ClientSecret,v_Data,"base64");
//
// retrieve Shopify's Hmac SHA256
v_ShopifyHash = "";
if(m_Payload.get("headers") != null)
{
if(m_Payload.get("headers").get("x-shopify-hmac-sha256") != null)
{
v_ShopifyHash = m_Payload.get("headers").get("x-shopify-hmac-sha256");
}
}
//
// build response to Shopify
m_ResponseHeader = Map();
m_ResponseHeader.put("status_code",401);
//
// compare our hash vs the received hash
if(v_VerifyHash == v_ShopifyHash)
{
m_ResponseHeader.put("status_code",200);
}
/*
//
sendmail
[
from :zoho.adminuserid
to :"This email address is being protected from spambots. You need JavaScript enabled to view it."
subject :"CLIENT Payload and Hash Comparison"
message :v_VerifyHash + "" + v_ShopifyHash + "" + m_Payload.toString()
]
//
// to cut even more time, we could store it directly in ZohoCRM and have a CRM workflow/automation forward it to Zoho Creator.
m_WebhookLog = Map();
m_WebhookLog.put("Name","Shopify - Inventory Update - " + zoho.currenttime.toString("yyyy-MM-dd HH:mm","Europe/London"));
m_WebhookLog.put("Body",m_Payload.get("body"));
m_WebhookLog.put("Event_Type","Inventory Update");
m_WebhookLog.put("Source","Shopify");
m_WebhookLog.put("Data_Format","JSON");
r_WebhookLog = zoho.crm.createRecord("Webhook_Payloads",m_WebhookLog);
*/
return {"crmAPIResponse":m_ResponseHeader};
- /* *******************************************************************************
- Function: fn_Shopify_Webhook_InventoryLevelUpdate(String crmAPIRequest)
- Label: Shopify - Webhook - Inventory Level Update
- Trigger: Function executed when a webhook from Shopify is sent to ZohoCRM (Inventory Update)
- Inputs: string crmAPIRequest
- Outputs: string crmAPIResponse
- ******************************************************************************* */
- //
- // need to respond in less than 1 second
- m_Payload = crmAPIRequest.toMap();
- m_Blank = Map();
- //
- // you created an app that generated a client ID and a client secret and resulted in an access token
- // the client secret used when generating the access token should be entered here
- v_ClientSecret = "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";
- //
- // store the record in ZohoCreator
- // I have a custom form in ZC with the fields "Event_Type" (single-line) and "JSON_Payload" (multi-line)
- m_CreateRecord = Map();
- m_CreateRecord.put("Event_Type","Order Update");
- m_CreateRecord.put("JSON_Payload",m_Payload.toString());
- r_CreateRecord = zoho.creator.createRecord("myOwnerName","myAppName","myFormName",m_CreateRecord,m_Blank,"myCreatorConnection");
- //
- v_Data = ifnull(m_Payload.get("body"),"");
- v_VerifyHash = zoho.encryption.hmacsha256(v_ClientSecret,v_Data,"base64");
- //
- // retrieve Shopify's Hmac SHA256
- v_ShopifyHash = "";
- if(m_Payload.get("headers") != null)
- {
- if(m_Payload.get("headers").get("x-shopify-hmac-sha256") != null)
- {
- v_ShopifyHash = m_Payload.get("headers").get("x-shopify-hmac-sha256");
- }
- }
- //
- // build response to Shopify
- m_ResponseHeader = Map();
- m_ResponseHeader.put("status_code",401);
- //
- // compare our hash vs the received hash
- if(v_VerifyHash == v_ShopifyHash)
- {
- m_ResponseHeader.put("status_code",200);
- }
- /*
- //
- sendmail
- [
- from :zoho.adminuserid
- to :"me+This email address is being protected from spambots. You need JavaScript enabled to view it."
- subject :"CLIENT Payload and Hash Comparison"
- message :v_VerifyHash + "" + v_ShopifyHash + "" + m_Payload.toString()
- ]
- //
- // to cut even more time, we could store it directly in ZohoCRM and have a CRM workflow/automation forward it to Zoho Creator.
- m_WebhookLog = Map();
- m_WebhookLog.put("Name","Shopify - Inventory Update - " + zoho.currenttime.toString("yyyy-MM-dd HH:mm","Europe/London"));
- m_WebhookLog.put("Body",m_Payload.get("body"));
- m_WebhookLog.put("Event_Type","Inventory Update");
- m_WebhookLog.put("Source","Shopify");
- m_WebhookLog.put("Data_Format","JSON");
- r_WebhookLog = zoho.crm.createRecord("Webhook_Payloads",m_WebhookLog);
- */
- return {"crmAPIResponse":m_ResponseHeader};
The Code Snippets: The Shopify Webhook Check and Restore
If you notice your webhooks automatically being removed. It's often because you have it processing data rather than responding straight away (less than 1 second requirement) to Shopify with "yes received" response.
As such, here's a function which lists the Shopify webhooks it's responsible for and if it finds one missing, it is to restore it. This function only has to check on 2 webhooks: the order update and the inventory level update:
/* ******************************************************************************* Function: String fn_Shopify_RestoreWebhook() Label: Fn - Shopify - Restore Webhook Trigger: Scheduled function to check on Shopify webhooks and to restore them if they have been removed. Inputs: - Outputs: - Date Created: 2024-04-22 (Joel Lipman) - Initial release Date Modified: ??? - ??? ******************************************************************************* */ // // set your own REST API endpoints for the webhooks here (we are only checking order update or inventory level update) v_CrmOrderUpdateWebhook = "https://www.zohoapis.com/crm/v2/functions/fn_shopify_webhook_orderupdate/actions/execute?auth_type=apikey&zapikey=1003.aaaabbbbccccddddeeeeffff00001111.22223333444455556666777788889999"; v_CrmInventoryLevelWebhook = "https://www.zohoapis.com/crm/v2/functions/fn_shopify_webhook_inventorylevelupdate/actions/execute?auth_type=apikey&zapikey=1003.aaaabbbbccccddddeeeeffff00001111.22223333444455556666777788889999"; // // restore by default b_RestoreOrderUpdateWebhook = true; b_RestoreInventoryLevelWebhook = true; // // retrieve connection details record r_ShopifyConnection = zoho.creator.getRecordById("myOwnerName","myAppName","myAPIConnectionsData",<myAPIConnectionRecordID>,"myCreatorConnection"); // // resolve connection details if(!isNull(r_ShopifyConnection.get("data"))) { m_Data = r_ShopifyConnection.get("data"); v_ShopifyID = ifnull(m_Data.get("Shop_ID"),""); v_APIVersion = ifnull(m_Data.get("API_Version"),""); v_AccessToken = ifnull(m_Data.get("Access_Token"),""); // m_Header = Map(); m_Header.put("X-Shopify-Access-Token",v_AccessToken); m_Header.put("Content-Type","application/json"); // // list all webhooks (note these are only webhooks created by apps, not added to Shopify manually) v_Endpoint = "https://" + v_ShopifyID + "/admin/api/" + v_APIVersion + "/webhooks.json"; r_ShopifyWebhooks = invokeurl [ url :v_Endpoint type :GET headers:m_Header ]; if(r_ShopifyWebhooks.get("webhooks") != null) { // // loop through to see if our custom notifications exist // note that installing other apps may create similar webhooks so you need to make sure the script below is looking for your custom webhook for each m_ThisWebhook in r_ShopifyWebhooks.get("webhooks") { if(m_ThisWebhook.get("id") != null) { // // my app is the only one using the orders/updated webhook topic/trigger if(m_ThisWebhook.get("topic").equalsIgnoreCase("orders/updated")) { b_RestoreOrderUpdateWebhook = false; } // // my app is the only one using the inventory_levels/update webhook topic/trigger if(m_ThisWebhook.get("topic").equalsIgnoreCase("inventory_levels/update")) { b_RestoreInventoryLevelWebhook = false; } } } } // // create order update webhook if(b_RestoreOrderUpdateWebhook) { m_Webhook = Map(); m_Webhook.put("format","json"); m_Webhook.put("topic","orders/updated"); m_Webhook.put("address",v_CrmOrderUpdateWebhook); m_Params = Map(); m_Params.put("webhook",m_Webhook); // v_Endpoint = "https://" + v_ShopifyID + "/admin/api/" + v_APIVersion + "/webhooks.json"; r_ShopifyOrderWebhook = invokeurl [ url :v_Endpoint type :POST parameters:m_Params.toString() headers:m_Header ]; } // // create inventory level update webhook if(b_RestoreInventoryLevelWebhook) { m_Webhook = Map(); m_Webhook.put("format","json"); m_Webhook.put("topic","inventory_levels/update"); m_Webhook.put("address",v_CrmInventoryLevelWebhook); m_Params = Map(); m_Params.put("webhook",m_Webhook); // v_Endpoint = "https://" + v_ShopifyID + "/admin/api/" + v_APIVersion + "/webhooks.json"; r_ShopifyInventoryWebhook = invokeurl [ url :v_Endpoint type :POST parameters:m_Params.toString() headers:m_Header ]; } } return r_ShopifyWebhooks;
- /* *******************************************************************************
- Function: string fn_Shopify_RestoreWebhook()
- Label: Fn - Shopify - Restore Webhook
- Trigger: Scheduled function to check on Shopify webhooks and to restore them if they have been removed.
- Inputs: -
- Outputs: -
- Date Created: 2024-04-22 (Joel Lipman)
- - Initial release
- Date Modified: ???
- - ???
- ******************************************************************************* */
- //
- // set your own REST API endpoints for the webhooks here (we are only checking order update or inventory level update)
- v_CrmOrderUpdateWebhook = "https://www.zohoapis.com/crm/v2/functions/fn_shopify_webhook_orderupdate/actions/execute?auth_type=apikey&zapikey=1003.aaaabbbbccccddddeeeeffff00001111.22223333444455556666777788889999";
- v_CrmInventoryLevelWebhook = "https://www.zohoapis.com/crm/v2/functions/fn_shopify_webhook_inventorylevelupdate/actions/execute?auth_type=apikey&zapikey=1003.aaaabbbbccccddddeeeeffff00001111.22223333444455556666777788889999";
- //
- // restore by default
- b_RestoreOrderUpdateWebhook = true;
- b_RestoreInventoryLevelWebhook = true;
- //
- // retrieve connection details record
- r_ShopifyConnection = zoho.creator.getRecordById("myOwnerName","myAppName","myAPIConnectionsData",<myAPIConnectionRecordID>,"myCreatorConnection");
- //
- // resolve connection details
- if(!isNull(r_ShopifyConnection.get("data")))
- {
- m_Data = r_ShopifyConnection.get("data");
- v_ShopifyID = ifnull(m_Data.get("Shop_ID"),"");
- v_APIVersion = ifnull(m_Data.get("API_Version"),"");
- v_AccessToken = ifnull(m_Data.get("Access_Token"),"");
- //
- m_Header = Map();
- m_Header.put("X-Shopify-Access-Token",v_AccessToken);
- m_Header.put("Content-Type","application/json");
- //
- // list all webhooks (note these are only webhooks created by apps, not added to Shopify manually)
- v_Endpoint = "https://" + v_ShopifyID + "/admin/api/" + v_APIVersion + "/webhooks.json";
- r_ShopifyWebhooks = invokeUrl
- [
- url :v_Endpoint
- type :GET
- headers:m_Header
- ];
- if(r_ShopifyWebhooks.get("webhooks") != null)
- {
- //
- // loop through to see if our custom notifications exist
- // note that installing other apps may create similar webhooks so you need to make sure the script below is looking for your custom webhook
- for each m_ThisWebhook in r_ShopifyWebhooks.get("webhooks")
- {
- if(m_ThisWebhook.get("id") != null)
- {
- //
- // my app is the only one using the orders/updated webhook topic/trigger
- if(m_ThisWebhook.get("topic").equalsIgnoreCase("orders/updated"))
- {
- b_RestoreOrderUpdateWebhook = false;
- }
- //
- // my app is the only one using the inventory_levels/update webhook topic/trigger
- if(m_ThisWebhook.get("topic").equalsIgnoreCase("inventory_levels/update"))
- {
- b_RestoreInventoryLevelWebhook = false;
- }
- }
- }
- }
- //
- // create order update webhook
- if(b_RestoreOrderUpdateWebhook)
- {
- m_Webhook = Map();
- m_Webhook.put("format","json");
- m_Webhook.put("topic","orders/updated");
- m_Webhook.put("address",v_CrmOrderUpdateWebhook);
- m_Params = Map();
- m_Params.put("webhook",m_Webhook);
- //
- v_Endpoint = "https://" + v_ShopifyID + "/admin/api/" + v_APIVersion + "/webhooks.json";
- r_ShopifyOrderWebhook = invokeUrl
- [
- url :v_Endpoint
- type :POST
- parameters:m_Params.toString()
- headers:m_Header
- ];
- }
- //
- // create inventory level update webhook
- if(b_RestoreInventoryLevelWebhook)
- {
- m_Webhook = Map();
- m_Webhook.put("format","json");
- m_Webhook.put("topic","inventory_levels/update");
- m_Webhook.put("address",v_CrmInventoryLevelWebhook);
- m_Params = Map();
- m_Params.put("webhook",m_Webhook);
- //
- v_Endpoint = "https://" + v_ShopifyID + "/admin/api/" + v_APIVersion + "/webhooks.json";
- r_ShopifyInventoryWebhook = invokeUrl
- [
- url :v_Endpoint
- type :POST
- parameters:m_Params.toString()
- headers:m_Header
- ];
- }
- }
- return r_ShopifyWebhooks;
The REST API endpoints
If you are having difficulty finding these, go to your CRM > Settings/Setup > Functions: then hover your mouse over the function and select "REST API". Enable the API Key option and copy the long URL to your clipboard for use in the above function:
Set up a schedule
If you find your webhooks being automatically removed by Shopify, then we put this schedule in to run as many times as possible during the day (but not enough to be banned):
- Login to ZohoCRM > Setup > Automation > Schedules > Create New Schedule
- I'm calling my schedule "Shopify - Check Webhooks" and I'm going to execute a writing function.
- It has a function name of "fn_Schedule_ShopifyWebhooks" and a display name of "Schedule - Shopify Webhooks"
- It has the code
- Starts ASAP; with a frequency of Hourly (run every 2 hours); and never ends.
- Save and done.
Error(s) Encountered
- webhook: expected String to be a Hash In this invokeURL, set the parameters to a JSON string using .toString().
- For those puzzled as to how you setup a webhook function in ZohoCRM: Create a function with a string parameter called "crmAPIRequest". It has to have this name. It is a reserved variable and will contain all the webhook payload data. When a webhook is pointed at it, the crmAPIRequest variable will hold the payload. Same with responding with the variable crmAPIResponse. Don't overthink it, just accept it. Less headaches for everyone.
Source(s):
- Shopify Dev - REST API - Retrieves a list of webhooks
- Shopify Dev - REST API - List of Webhook event topics
- Shopify Dev - REST API - HTTPS webhook delivery
- Wikipedia - List of HTTP status codes
- Zoho CRM Developer Docs - Serverless Functions - Request and Response Object
- Joel Lipman - ZohoCRM & Xero Real-Time Invoices: Receive Webhook