All SQL topics
SQL TOPIC

SQL text functions: clean messy data

LOWER, TRIM, SUBSTR and REPLACE tidy up text so duplicates and groups line up.

LOWER(TRIM(email)) makes "ANNA@EXAMPLE.DE " and "anna@example.de" the same value, which is how you find duplicate customers.

SUBSTR(email, INSTR(email, '@') + 1) extracts the email domain. Use || to join text together.

Worked example

Normalise customer emails.

SELECT name, LOWER(TRIM(email)) AS clean_email
FROM customers
WHERE email <> LOWER(TRIM(email));
Result on the sample data
nameclean_email
Anna Schmidtanna.schmidt@example.de

Common mistakes

  • Cleaning only in the SELECT, then grouping by the raw column.
  • Forgetting that trailing spaces make two values different.