Checking for duplication

Duplication can be checked on String, Number, and Date fields by setting the parameter is_unique:

In additional, Uniqueness rules can be set up which check for uniqueness across multiple sets of columns. These have the rule_type="Uniqueness" and automatically quarantine any data that does not meet the criteria.

They take the following parameters:

Note that "rule_type": "Uniqueness" does not use the "field" parameter:

config = {"rules": [
	{
		"rule_type": "Uniqueness",
		"params": {
			"unique_fields": "device_id,date"
		},
	}
]}