Zoho Analytics & Zoho People: Monitor DataSource Sync

: Joel Lipman; Zoho; 02 July 2024; Hits: 1513

What?
A quick article on how to setup an automation that checks if a datasource synchronization between Zoho People and Zoho Analytics failed. This can be adapted to any data source for Zoho Analytics.

Why?
We're assisting a client implement Zoho People as their new HR solution. Zoho Analytics is used to generate specific spreadsheets for a feed into their third-party payroll system. If the data is incorrect in Zoho Analytics, and payroll gets run with these discrepancies, then they will have a fair few disgruntled employees who may have been paid incorrectly. As such this system has to be reliable and the system managers need to be notified in advance of any errors with the intention to be able to correct any inconsistencies themselves.

How?
We're going to add a scheduled workflow on a daily basis at 1am in the morning every day. This will be added to the client's Zoho People instance which will check the datasource metadata via the Analytics API.

Add a custom scheduler in Zoho People

Login to ZohoPeople as a system administrator
Click on the cog icon in the top right to go to the setup screen.
Under "Automation", click on "Scheduler"
Click on the "Add Scheduler" button
Complete the "Scheduler" header information such as the name, description, start date/time, frequency, and timezone.
While you're here, click on "Connections" at the top of the IDE (where you paste the code) and ensure you setup a Connection that does NOT use the credentials of the logged-in user and has as scope ZohoAnalytics.metadata.read.

The code

copy raw

/* *******************************************************************************
Function:       void Check_DataSources_Sync_to_ZohoAnalytics ()
Label:          Check DataSources Sync to ZohoAnalytics
Trigger:        Scheduled daily at 1am starting from tomorrow (3-Jul-2024)
Purpose:		To warn system managers if a sync started failing.
Inputs:         None
Outputs:        Emails system manager if any failures.

Date Created:   2024-07-02 (JoelLipman.com - Joel Lipman)
                - Initial release
Date Modified:	???
                - ???

More Information:
                https://www.zoho.com/analytics/api/v2/metadata-api/datasources.html

******************************************************************************* */
//
// initialize
v_LastSyncTime = null;
v_LastSyncStatus = "?";
m_DataSourceInfo = Map();
//
// read the TLD (eu or com or com.in) from the URL when logged into Zoho Analytics
v_BaseURL = "https://analyticsapi.zoho.com";
//
// to get workspace ID, look at the URL in ZohoAnalytics and take the number straight after the word "workspace/"
v_WorkspaceID = "12345000001234567";
//
// to get the Analytics Org ID, login to Zoho Analytics, ensuring you are on the "Home" page (click on "Home" in top menu if not),
// then click on the cog icon "Organization Settings", then click on "Organization Details",
// then look at the URL (website address) and take the number directly after "org-details/"
v_AnalyticsOrgID = "20240702123";
//
// generate request header
m_Header = Map();
m_Header.put("ZANALYTICS-ORGID",v_AnalyticsOrgID);
//
// generate the endpoint to read the analytics metadata on the syncs
v_Endpoint = v_BaseURL + "/restapi/v2/workspaces/" + v_WorkspaceID + "/datasources";
//
// send the request
r_Analytics = invokeurl
[
	url :v_Endpoint
	type :GET
	headers:m_Header
	connection:"my_connection"
];
//info r_Analytics;
//
// parse response
if(r_Analytics.get("status") == "success")
{
	m_ResponseData = ifnull(r_Analytics.get("data"),Map());
	l_DataSources = ifnull(m_ResponseData.get("dataSources"),List());
	for each  m_DataSource in l_DataSources
	{
		if(m_DataSource.get("datasourceName").equalsIgnoreCase("Zoho People"))
		{
			v_LastSyncTime = m_DataSource.get("lastDataSyncTime");
			v_LastSyncStatus = m_DataSource.get("lastDataSyncStatus");
			m_DataSourceInfo = m_DataSource;
			break;
		}
	}
}
//info m_DataSourceInfo;
//
// to email or not to email, that is the question, whether 'tis nobler in the mind to suffer...
if(!v_LastSyncStatus.equalsIgnoreCase("Data Sync Successful"))
{
	v_To = "Client <my_client@their_company.com>";
	v_Cc = {"Joel Lipman Support Desk <This email address is being protected from spambots. You need JavaScript enabled to view it.>"};
	v_Subject = "URGENT: Sync Between Zoho People and Zoho Analytics Failed!!!";
	v_Message = "Hi there!<br /><br />This is an email to advise that the synchronization between your Zoho People and Zoho Analytics instances failed.  The last synchronization data we have is as follows:<br /><br /><table style='margin:0 auto;'>";
	if(!isNull(m_DataSourceInfo.get("datasourceName")))
	{
		for each  v_DataSourceKey in m_DataSourceInfo.keys()
		{
			v_Message = v_Message + "<tr><td>" + v_DataSourceKey + "</td><td>" + ifnull(m_DataSourceInfo.get(v_DataSourceKey),"?") + "</td></tr>";
		}
	}
	else
	{
		v_Message = v_Message + "<tr><td><b>No Datasource Information!</b> Either the data source synchronization has been switched off or there is an error with a connection account between Zoho People and Zoho Analytics.</td></tr>";
	}
	v_Message = v_Message + "</table><br />This may have been a one-off blip but if you get this error again tomorrow, you should raise this with either Zoho or your Zoho partner.  This email will also have been copied to the ticket support system of your Zoho Partner.<br /><br />Kind Regards,<br /><br /><b>the Joel Lipman team</b><br /><a href='mailto:This email address is being protected from spambots. You need JavaScript enabled to view it.'>This email address is being protected from spambots. You need JavaScript enabled to view it.</a><br /><a href='https://joellipman.com'>www.joellipman.com</a>";
	sendmail
	[
		from :zoho.adminuserid
		to :v_To
		cc:v_Cc
		subject :v_Subject
		message :v_Message
	]
}

/* *******************************************************************************
Function: void Check_DataSources_Sync_to_ZohoAnalytics ()
Label: Check DataSources Sync to ZohoAnalytics
Trigger: Scheduled daily at 1am starting from tomorrow (3-Jul-2024)
Purpose: To warn system managers if a sync started failing.
Inputs: None
Outputs: Emails system manager if any failures.
Date Created: 2024-07-02 (JoelLipman.com - Joel Lipman)
- Initial release
Date Modified: ???
- ???
More Information:
https://www.zoho.com/analytics/api/v2/metadata-api/datasources.html
******************************************************************************* */
//
// initialize
v_LastSyncTime = null;
v_LastSyncStatus = "?";
m_DataSourceInfo = Map();
//
// read the TLD (eu or com or com.in) from the URL when logged into Zoho Analytics
v_BaseURL = "https://analyticsapi.zoho.com";
//
// to get workspace ID, look at the URL in ZohoAnalytics and take the number straight after the word "workspace/"
v_WorkspaceID = "12345000001234567";
//
// to get the Analytics Org ID, login to Zoho Analytics, ensuring you are on the "Home" page (click on "Home" in top menu if not),
// then click on the cog icon "Organization Settings", then click on "Organization Details",
// then look at the URL (website address) and take the number directly after "org-details/"
v_AnalyticsOrgID = "20240702123";
//
// generate request header
m_Header = Map();
m_Header.put("ZANALYTICS-ORGID",v_AnalyticsOrgID);
//
// generate the endpoint to read the analytics metadata on the syncs
v_Endpoint = v_BaseURL + "/restapi/v2/workspaces/" + v_WorkspaceID + "/datasources";
//
// send the request
r_Analytics = invokeUrl
[
url :v_Endpoint
type :GET
headers:m_Header
connection:"my_connection"
];
//info r_Analytics;
//
// parse response
if(r_Analytics.get("status") == "success")
{
m_ResponseData = ifnull(r_Analytics.get("data"),Map());
l_DataSources = ifnull(m_ResponseData.get("dataSources"),List());
for each m_DataSource in l_DataSources
{
if(m_DataSource.get("datasourceName").equalsIgnoreCase("Zoho People"))
{
v_LastSyncTime = m_DataSource.get("lastDataSyncTime");
v_LastSyncStatus = m_DataSource.get("lastDataSyncStatus");
m_DataSourceInfo = m_DataSource;
break;
}
}
}
//info m_DataSourceInfo;
//
// to email or not to email, that is the question, whether 'tis nobler in the mind to suffer...
if(!v_LastSyncStatus.equalsIgnoreCase("Data Sync Successful"))
{
v_To = "Client <my_client@their_company.com>";
v_Cc = {"Joel Lipman Support Desk <This email address is being protected from spambots. You need JavaScript enabled to view it.>"};
v_Subject = "URGENT: Sync Between Zoho People and Zoho Analytics Failed!!!";
v_Message = "Hi there! This is an email to advise that the synchronization between your Zoho People and Zoho Analytics instances failed. The last synchronization data we have is as follows: <table style='margin:0 auto;'>";
if(!isNull(m_DataSourceInfo.get("datasourceName")))
{
for each v_DataSourceKey in m_DataSourceInfo.keys()
{
v_Message = v_Message + "<tr><td>" + v_DataSourceKey + "</td><td>" + ifnull(m_DataSourceInfo.get(v_DataSourceKey),"?") + "</td></tr>";
}
}
else
{
v_Message = v_Message + "<tr><td>No Datasource Information! Either the data source synchronization has been switched off or there is an error with a connection account between Zoho People and Zoho Analytics.</td></tr>";
}
v_Message = v_Message + "</table> This may have been a one-off blip but if you get this error again tomorrow, you should raise this with either Zoho or your Zoho partner. This email will also have been copied to the ticket support system of your Zoho Partner. Kind Regards, the Joel Lipman team <a href='mailto:This email address is being protected from spambots. You need JavaScript enabled to view it.'>This email address is being protected from spambots. You need JavaScript enabled to view it.</a> <a href='https://joellipman.com'>www.joellipman.com</a>";
sendmail
[
from :zoho.adminuserid
to :v_To
cc:v_Cc
subject :v_Subject
message :v_Message
]
}