The Ins and Outs of Merge/Purge for Email

The term merge/purge (merge purge) or dedupe has a long history that has its roots in pre-internet age direct marketing. A common situation for an advertising using direct mail was the need to regularly obtain new recipient names and addresses to add to their database, in order to acquire new customers and grow their businesses. This was generally accomplished by renting or buying mailing lists from 3rd party companies, sometimes called list brokers. A large industry grew over many years, based on compiling large databases of consumer names and addresses (along with plenty of other demographic, psychographic, and other data) and making them available for sale or rental to catalog companies, retailers, and many types of direct marketing companies.

While adding new prospects to a company’s database has always been a goal for businesses looking to grow, the practice of adding names in large quantities also creates challenges. One of the most basic is the need to identify and remove duplicate records among a number of files, to be used in a marketing campaign. This is where the merge/purge process largely came into being. A company renting a list from a 3rd party would need a way to compare the new list against their internal customer and/or prospect file, identify duplicates, and remove them from a new file of unique records that do not appear on their internal files. This reduced waste in sending duplicate direct mail to the same recipient, and perhaps to avoid sending an acquisition offer to a current customer. There are many use cases for merge/purge to control what records appear on a final marketing campaign file. One other common usage when comparing a number of lists, is to identify those names that appear on multiple lists. These individuals can often be more responsive than unique records, as their appearance on multiple lists may indicate that they have engaged (i.e. purchased) from multiple companies.

In the direct mail industry, the merge/purge process is typically handled by the mailer’s service bureau. The results of the merge/purge with regard to rented mailing lists are often used to set the final rental price, as it may be based on the number of new or unique names provided to the list renter.

While the merge/purge process is a staple of the direct mail business, it is also used in numerous other types of direct marketing, such as email marketing. In email marketing, a merge/purge can be used in a very similar manner to direct mail. Advertisers may own multiple internal lists of email addresses (current customers, past customers, prospects, etc.) that may or may not be stored in a central database. If they are maintained as separate lists, there are likely to be times when the company needs to compare the lists identifying email address records that appear on multiple lists and making any necessary updates to those lists (for example, removing an address from a prospect list, once it appears on a customer list). Additionally, if an advertiser obtains new email records through email list rental, purchase or some other manner, the company will want to dedupe those 3rd party lists against their own house files. Unlike in the direct mail industry, there aren’t a large number of outside companies designed to provide a neutral platform where the email list merge/purge can occur. This is why OPTIZMO developed COMPARE, as a tool to meet this need in the email marketing industry.

If you have a regular or occasional need to compare and dedupe multiple email lists – either in-house or with 3rd parties – you should contact us to see if we can provide a simple, cost-effective solution.