When performing a restore of a Dynamics database, it is important to note that in certain scenarios Email Sends which have completed may run again after the database restore is complete, while Campaign Automations may encounter conflicts if not paused during the copy/restore process. This article goes into the details of those behaviors and provides steps for the copy/restore process.
WARNING: When you import a copy of your production environment into your Sandbox, all connection details remain the same as in production. If plugins are still enabled in Sandbox, anything deleted from the Sandbox will be deleted from the production account on our cloud and will be irrevocably inaccessible from the production environment.
Understanding Scheduled Email Sends
Email Sends which are scheduled to run in the future use workflows in Dynamics. These workflows wait until the date/time they are supposed to execute. Once the date/time is reached, the workflow proceeds with the email send. These email sends can be viewed by going to Advanced Settings > System Jobs in Microsoft Dynamics. The workflow will be in a 'waiting' state.
Take this scenario for example:
- On January 1st an email send is scheduled to run on January 14th. A Dynamics database backup is taken on January 7th. January 14th arrives and the email send executes as desired.
- On January 17th, a user accidentally bulk deletes records in Dynamics. It is determined by the system administrator that a restore of the Dynamics database is the easiest method to recover the data. The administrator restores the Dynamics database backup from the 7th.
- When the database is restored, Dynamics sees that there is a 'ClickDimensions Execute Send' workflow which was scheduled to run on January 14th, which was three days ago. Dynamics immediately executes the workflow, causing the email send which ran prior to the database restore to run again.
When a backup is taken, any Execute Send workflows which have not met the 'send on' date will run if the database is restored AFTER that date (i.e., the Workflow instance is in a 'Waiting' state in the backup of the Dynamics database). These 'ClickDimensions Execute Send' workflows ultimately make Dynamics communicate with Click's servers via the internet.
Email Send Considerations
When performing a restore of a Dynamics database, it’s important to be aware of any Email Sends which may be in a waiting state in the backed up database. These will either execute when the database is restored OR remain in a waiting state. This will depend on the date of the backup, the date of the restore, and the date the Email Sends are scheduled to take place.
A way to prevent an Email Send from running from a restored database which has not run yet is to cancel the waiting workflow. This can be done in Dynamics by going to Advanced Settings > System Jobs and canceling any 'ClickDimensions Execute Send' workflows in a waiting state. This would be the case if you are performing a Dynamics Online restore to a Sandbox instance and don’t want these to run in both environments.
For Email Sends which were scheduled to run at a particular date and time which is prior to the restore, these workflows will run immediately when the Dynamics organization is restored and enabled.
Campaign Automation Considerations
Click's Campaign Automation service relies exclusively on the automation ID to track and execute automations. In contrast, most of our other services, such as Email and Forms, utilize both the account key and record ID. When performing a database copy, the entire record, including its ID, is duplicated. Within our back-end service, this record is stored with the same ID and account key.
Under normal circumstances, this does not present an issue, as other services reference both identifiers. However, because Campaign Automation relies solely on the automation ID, it retrieves responses from both the Sandbox and Production environments without differentiation, which results in conflicts causing problems with the Campaign Automation progression. In this situation where there are two environments containing Campaign Automations with matching automation IDs, if Campaign Automations are deleted from the Sandbox environment, the corresponding entries/IDs will be deleted from Click's back-end service, leaving only the Production IDs - this resolves conflicts arising from the duplicated record ID and allows the Production Campaign Automations to resume normal operation.
Recommended Process
-
Pause Active Campaign Automations and Halt Incoming Activities in Production:
- Ensure that all active Campaign Automations are paused. Additionally, confirm that no emails are being sent, and no form submissions are being received in the production environment during this period. This prevents any unintended data processing during the backup.
- Ensure that all active Campaign Automations are paused. Additionally, confirm that no emails are being sent, and no form submissions are being received in the production environment during this period. This prevents any unintended data processing during the backup.
-
Review and Address 'ClickDimensions Execute Send' Workflows:
- Before creating a copy of your production database, navigate to Advanced Settings > System Jobs in Dynamics to identify any 'ClickDimensions Execute Send' workflows in a waiting state. Allow these workflows to complete or cancel them to prevent unintended email sends post-restore.
- Before creating a copy of your production database, navigate to Advanced Settings > System Jobs in Dynamics to identify any 'ClickDimensions Execute Send' workflows in a waiting state. Allow these workflows to complete or cancel them to prevent unintended email sends post-restore.
-
Disable Plugins and Workflows in the Sandbox Environment:
- Prior to initiating the database copy, ensure that all plugins and workflows in the sandbox environment are disabled. This precaution prevents any automatic processes from triggering during or immediately after the database restoration.
- Prior to initiating the database copy, ensure that all plugins and workflows in the sandbox environment are disabled. This precaution prevents any automatic processes from triggering during or immediately after the database restoration.
-
Perform the Database Copy:
- Proceed with the database copy from the production environment to the sandbox environment. Ensure that the copy process is executed during a maintenance window to minimize potential disruptions.
- Proceed with the database copy from the production environment to the sandbox environment. Ensure that the copy process is executed during a maintenance window to minimize potential disruptions.
-
Register the Sandbox for its own Click solution
- Once you have made the copy to a sandbox org, register the sandbox org for its own Click solution and install it it into your sandbox. This will overwrite the production Account Key that is embedded throughout Dynamics.
- Once you have made the copy to a sandbox org, register the sandbox org for its own Click solution and install it it into your sandbox. This will overwrite the production Account Key that is embedded throughout Dynamics.
-
Import the Click Solution in the Test Environment:
- After the database copy, import the Click solution specific to the sandbox environment. During the import, select the UPDATE & OVERWRITE CUSTOMIZATIONS option. This step is crucial to overwrite the production Account Key embedded in the database with the sandbox environment's Account Key, ensuring proper integration and preventing cross-environment data issues.
- After the database copy, import the Click solution specific to the sandbox environment. During the import, select the UPDATE & OVERWRITE CUSTOMIZATIONS option. This step is crucial to overwrite the production Account Key embedded in the database with the sandbox environment's Account Key, ensuring proper integration and preventing cross-environment data issues.
-
Maintain Disabled State of Plugins and Workflows; Address Copied Records:
- Keep the plugins and workflows disabled. Carefully review and delete any records that were copied over from production, especially those that could trigger actions in the production environment if processed. Since the plugins and workflows are disabled, this deletion should not affect the production data.
- Keep the plugins and workflows disabled. Carefully review and delete any records that were copied over from production, especially those that could trigger actions in the production environment if processed. Since the plugins and workflows are disabled, this deletion should not affect the production data.
-
Re-enable Plugins and Workflows:
- Once the necessary records have been addressed and you're confident that enabling the plugins and workflows won't cause unintended actions, re-enable them in the sandbox environment.
- Once the necessary records have been addressed and you're confident that enabling the plugins and workflows won't cause unintended actions, re-enable them in the sandbox environment.
-
Re-authenticate the Service User in the Sandbox Environment:
- Ensure that the Service User account in the sandbox environment is re-authenticated. This step is essential to maintain proper communication between Dynamics and Click's services.
Additional Recommendations:
-
Utilize Click's Export/Import Tools for Specific Records:
- For entities such as Email Templates, Forms, Surveys, and Landing Pages, consider using Click's Export and Import tools to move these records between environments. This method ensures that the records are correctly linked to the appropriate environment and prevents potential issues arising from direct database copies.
- For entities such as Email Templates, Forms, Surveys, and Landing Pages, consider using Click's Export and Import tools to move these records between environments. This method ensures that the records are correctly linked to the appropriate environment and prevents potential issues arising from direct database copies.
-
Coordinate with Click Support:
- If you encounter any uncertainties or require assistance during this process, it's advisable to contact Click Support. We can provide guidance tailored to your specific scenario and ensure that best practices are followed.