Messages passed from remote server to `deliveries.output` are not sanitized

[olegbliaher]

Describe the bug

Started seeing the following issue in periods in Sentry:

internal error: Mysql2::Error: Incorrect string value: '\xD3\xCA\xCF\xE4\xB2\xBB...' for column `postal-server-123`.`deliveries`.`output` at row 1

When I checked the specific message in the GUI, I see that it's being logged as an Error, instead of a hard fail. The trigger for this is someone misspelling their email address as GNAIL instead of GMAIL. gnail.com actually has a proper MX, a Chinese one, which returns "mailbox does not exist" in English and Chinese.

The Chinese characters are not considered to be valid characters.

To Reproduce

Just send a test email to someone@gnail.com

Expected behaviour

A proper hard fail with mailbox does not exist, no retries. I tried telnetting, the remote server does return a 550.

Environment

Postal 3.3.7 docker on a Debian 12 host.

[willpower232]

Can you confirm what database charset you are using?

[olegbliaher]

@willpower232 utf8mb4

or, to be more specific:

DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci