When the ClickDimensions solution is installed, it adds custom entities to your Dynamics 365 (D365) database and UI. These entities are provided to allow your users to access ClickDimensions features as well as to store data generated by the integration. Two primary data drivers can potentially result in large volumes of data—email and web analytics.
This article will cover the methods by which these records are generated, as well as some of our suggestions for how long data should be retained before deletion to help prevent an over-abundance of data from slowing down your system. Please note that these are only suggestions and you may decide to keep records for a longer or shorter period according to your organization's needs.
Email Analytics
- Sent Emails (cdi_sentemail)
Store a summary of interactions for each recipient of an Email Send. We recommend retaining these records for 6 months. - Email Events (cdi_emailevent)
Store the granular interactions. One Email Event record is created for each interaction—opens, bounces, clicks, etc.—with an email. We recommend retaining these records for 6 months. You can also reach out via Support ticket to request that the syncing of Email Events for Delivery, Open, Click, Bounce or Spam events be disabled. This will prevent those types of Email Event records from syncing to CRM to save space, and you can still check the number of these events by viewing the Email Statistics within an Email Send record. - Unsubscribes (cdi_unsubscribe)
Hold global unsubscribe data whether that data has been imported from a legacy email marketing system or generated over time using ClickDimensions. These records should be kept to respect the subscription preferences of your recipients. - Excluded Recipients (cdi_excludedemail)
Store why a recipient was excluded from an Email Send—previous bounces, an unsubscribe (or opt-out preference), a missing email address, etc.. We recommend retaining these records for 6 months. - Posted Preferences (cdi_subscriptionpreference)
Store a recipient’s opt-in and opt-out preferences for your subscription lists. These records should be kept to respect the subscription preferences of your recipients and to confirm their subscriptions.
Sample of Email analytics data
The amount of data generated will vary depending on usage. For example, if an Email Send has 100,000 recipients, then 100,000 Sent Email (or Excluded Recipients) records will be created. Email Events records for bounces, opens, clicks, and more would also be created, with each record containing one interaction per a recipient.
So if that Email Send results in a 25% open rate, 4% bounce rate, and 12% click-through rate, approximately 140,000 total new records would be added to the D365 environment the email was sent from.
| Sent Email (or Excluded Recipients) records | 100,000 |
| Email Events (bounces) | 4,000 |
| Email Events (clicks) | 12,000 |
| Email Events (opens) | 25,000 |
| Unsubscribes | 500 |
| Total records created | 141,500 |
If your company plans on sending weekly emails to 100,000 recipients, this could result in over 560,000 new records being added to D365 each month.
Web Analytics
As part of our 12.3 release, ClickDimensions will no longer sync Anonymous Visitor records or their unidentified Page Views, Visits, or IP Organizations to your CRM. Once an Anonymous Visitor is identified, then the related Page Views, Visits, and IP Organizations will be synced back to CRM. If your organization needs these records for some reason, then you can open a Support ticket to request the syncing of these records, but this will cause a significant increase in the numbers of records generated.
- IP Organizations (cdi_iporganization)
Stores the IP address of visitors to your site; not duplicated if multiple visitors share an IP. We recommend keeping these records as multiple visitors may be tied to a single IP Organization. - Anonymous Visitors (cdi_anonymousvisitor)
Stores unknown visitors from your site. As of our 12.3 release, these records will no longer sync by default but this syncing can be re-enabled. If your organization does not make use of these records, then the historical records prior to release 12.3 can be safely deleted. - Visits (cdi_visit)
Stores each visit by a Contact or Lead. We recommend retaining these records for 3 months. - Page Views (cdi_pageview)
Stores each page on your site which a visitor has viewed. We recommend retaining these records for 3 months.
Sample of Web Analytics data
For example, let's say your site attracts 10,000 visits per day with most visitors originating from different IP addresses and each visitor views an average of 1.5 pages per visit. If each visitor only visits your site once a month, then more than 1 million total records per month would be created in your CRM.
| IP Organization records (95% from different IPs x 30 days) | 285,000 |
| Visit records (10,000 visitors/day x 30 days) | 300,000 |
| Page Views (10,000 visitors/day x 1.5 pages viewed x 30 days) | 450,000 |
| Total records created in 1 month | 1,035,000 |
TIP—for high-traffic sites, place the ClickDimensions tracking script only on pages that are most important for your sales and marketing users to see within CRM. (For example, a page representing a PPC ad conversion.) Placing the script globally would record every single page view, regardless of importance, and potentially flood your CRM with too many records to process.
Additional Types of Data
While these types won't add as much data as web analytics or email, they are still worth noting so you're aware of what data can be created.
- Forms and Surveys
A Posted Form/Survey record will be created in D365 for each submission along with a Posted Field or Survey Answer record for each field on the Form or Survey. - Event Connectors
Event records are created for each event from a connector (GoToWebinar, Eventbrite, Webex, and Cvent). Event Participation records for each participant who registers for an event. - Campaign Automations
Participants receiving an email from a Campaign Automation will have an individual Email Send record created. (If many participants go through the Send Email action at the same time, they will all be on the same Email Send record.) Sent Emails, Email Events, and Unsubscribes would be created as expected with a regular Email Send. - SMS messages
Messages sent through ClickDimensions or incoming replies to them will be recorded as Text Message activity records.
Database Size Growth
Keep in mind that your D365 database will grow in size over time and the rate at which it grows will vary depending on your overall usage of D365. ClickDimensions, other integrated applications, and your maintenance plan (i.e., bulk delete jobs) will all impact that growth.
As a rule of thumb, D365 records created by ClickDimensions will take up 1kb-2kb per record in disk storage space. Using the examples cited above (sending a weekly bulk email to 100,000 recipients and tracking 10,000 website visitors per day), you could expect to see approximately 1,600,000 new records created a month. Meaning approximately 1.6GB to 2.4GB of storage space needed each month. So please plan in advance which data (especially web tracking) is important to capture, which data you will keep, and how long you will keep it.