Decide what counts as a match
For a person, a verified email address is often a strong candidate, but shared inboxes and changed addresses create exceptions. Phone numbers can be useful after normalization, yet family or office numbers may be shared. For an organization, a legal name alone is weak because abbreviations and subsidiaries differ. A domain, external customer ID, or verified account number can strengthen the match.
Write the policy before writing code:
Never silently choose the first result when the search returns several plausible records.
| Entity | Strong signal | Review case |
|---|
| Person | Exact normalized email | Shared inbox or no email |
| Person | External customer ID | ID reused across systems |
| Organization | Exact domain plus similar name | Holding company or subsidiary |
| Organization | External account ID | Imported legacy IDs |
Normalize at the boundary
Normalize only for comparison; preserve the user’s preferred display value for the record. Lowercase email domains and trim surrounding whitespace. Use a proper email parser rather than deleting every punctuation mark. Convert phone numbers to a consistent international representation only when the country context is known.
For organizations, collapse harmless whitespace and compare a separate normalized name, but do not remove meaningful legal suffixes blindly. “North Star LLC” and “North Star Holdings” may be different entities. Keep both the raw value and comparison value in your integration’s audit record.
Search before create
The flow should be:
The mapping table is important because a search endpoint may be expensive, paginated, or affected by formatting. A stable source-to-Pipedrive mapping turns future updates into direct PATCH operations.
- Receive the source record and external ID.
- Check your own mapping table by source system and ID.
- If no mapping exists, search Pipedrive using the strongest available identifier.
- Score candidates and require review for ambiguous matches.
- Create only when no candidate meets the threshold.
- Save the Pipedrive ID and source ID together.
The race condition
Two workers can both search, see no match, and create two people. A process-local lock is not enough when you have multiple instances. Use a database uniqueness constraint on source system plus external ID, or a short-lived distributed lock keyed by the normalized identity. The lock must cover the search and create decision.
If the create succeeds but the worker crashes before saving the mapping, a retry can still search and find the just-created record. That is why the search must happen again and why a deterministic external ID is valuable when your source system supports it.
Retry safely
A timeout does not tell you whether Pipedrive accepted the create. Before retrying a create, search by the strongest identity and inspect recent candidates. Do not blindly replay a POST. If the API or endpoint supports an idempotency mechanism, use it and persist the key across retries.
Log an internal operation ID, source ID, candidate IDs, decision, and final Pipedrive ID. Redact email addresses and phone numbers in ordinary logs. Store detailed matching evidence only where access is controlled.
Link people and organizations carefully
An organization match can be uncertain even when a person match is strong. Resolve the person and organization independently, then update the relationship. Do not create a new organization simply because the person’s incoming company name differs by punctuation. Flag a likely alias for review.
When a source record changes company, preserve history according to your business rules. Reassigning a person may be correct; overwriting an organization on every sync can destroy useful CRM context. Separate identity resolution from relationship policy.
Use a confidence score, not a magic guess
A practical scoring model can award points for exact external ID, exact normalized email, exact domain, and similar name, then subtract points for conflicting owner or geography. The threshold must be tested on real historical data. A score is an aid to consistent judgment, not a fact about identity.
Keep three outcomes: match, create, and review. The review queue is a feature. It is cheaper than repairing hundreds of merged or duplicated records later.
Reconcile after the live sync
Run a periodic report for repeated normalized emails, organizations sharing a domain, and records created by the same job within a short interval. Investigate rather than automatically deleting. Soft-deleted records and permissions can affect what a token can see.
Measure duplicate rate, ambiguous-match rate, create retries, and mapping failures. If duplicates rise after a deployment, compare changes in normalization, query limits, and concurrency before changing the matching threshold.
Decide what humans should see
An operator needs enough evidence to approve or reject a candidate without opening five systems. Show the source ID, masked email or domain, candidate Pipedrive IDs, matching signals, conflicting values, and the proposed action. Do not display a full phone number or personal record to everyone who can access the queue.
When a reviewer approves a match, save the decision and rule version. When a reviewer rejects it, preserve the reason and prevent the same ambiguous input from creating an endless stream of new candidates. A review queue should be finite and auditable.
For imports, preview the create and match counts before execution. A preview that says 480 creates and 12 ambiguous records gives the owner a chance to fix source data before those records reach the CRM. After execution, export the mapping results and compare them with the source count.
If a duplicate is discovered, do not automatically delete one record. Identify the canonical record, preserve activities and ownership according to policy, and document the merge or correction. Deletion can remove context that another team still needs.
The trap: search-before-create without a uniqueness boundary is not duplicate prevention. It only reduces duplicates when one worker is acting alone.
The minimum duplicate test set
Test an exact external-ID match, normalized email match, name-only candidate, two concurrent creates, a timeout after create, and a retry after partial failure. Record whether the expected result is update, skip, review, or create. This turns “search before create” into a deterministic policy rather than a best-effort lookup that behaves differently across workflows.
Where these facts come from
