Dedupe

<< Click to Display Table of Contents >>

Navigation:  Reference > Transforms >

Dedupe

Description

Remove duplicate rows.

 

Example

Keep the first record for each unique 'Customer Id':

how to dedupe a dataset example

 

Inputs

One.

 

Options

Check the column(s) you wish to look for duplicate values in.

Check case sensitive to use case sensitive matching.

 

Notes

Rows are considered duplicates if they have exactly the same value in all the checked columns.

Comparisons are whitespace sensitive. You can use Whitespace to remove whitespace before deduping and Replace to get of other unwanted characters (e.g. whitespace inside the text).

When several rows are duplicates, only the top one is retained.

The Unique transform is a more powerful (but more complex and slower) alternative to Dedupe.

 

See also

Dedupe a dataset