Skip to content

Fix GH-23444: pdo_odbc does not convert Unicode data outside Windows - #23445

Open
lazerg wants to merge 1 commit into
php:PHP-8.4from
lazerg:fix/gh-23444-pdo-odbc-utf8-conv
Open

Fix GH-23444: pdo_odbc does not convert Unicode data outside Windows#23445
lazerg wants to merge 1 commit into
php:PHP-8.4from
lazerg:fix/gh-23444-pdo-odbc-utf8-conv

Conversation

@lazerg

@lazerg lazerg commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

PDO::ODBC_ATTR_ASSUME_UTF8 is documented as Windows only, and the UTF-8 conversion behind it sits inside #ifdef PHP_WIN32. pdo_odbc_sqltype_is_unicode() was not guarded the same way, so elsewhere wide parameters and columns were still bound as SQL_C_BINARY and then passed through unconverted. Raw UTF-8 reached the server for an nvarchar parameter and raw UTF-16 came back for an nvarchar column. With msodbcsql18 a parameter of odd byte length is rejected with HY090.

Report wide types as not Unicode outside Windows, which leaves the encoding to the driver as the default already does. Patch by @NattyNarwhal.

Verified against SQL Server 2025 and ODBC Driver 18 through unixODBC 2.3.14 on macOS. ext/pdo_odbc/tests is green with that DSN and unchanged against the SQLite3 ODBC driver.

The N'…' literal in the reported snippet is part of the statement text, which the driver manager widens byte by byte on its way to a Unicode-only driver. That is independent of the bound parameter and is not addressed here.

Fixes GH-23444

@lazerg
lazerg force-pushed the fix/gh-23444-pdo-odbc-utf8-conv branch 2 times, most recently from 8269a2e to b169d2b Compare August 25, 2026 10:09
@NattyNarwhal

NattyNarwhal commented Aug 25, 2026

Copy link
Copy Markdown
Member

I think this is overkill for an older release branch. It might be the right approach in master if drivers really prefer taking wchars, but I suspect that on Unix it doesn't make sense (most drivers handle UTF-8 binding and conversion fine it seems if you tell them to, and wchar can be variable depending on i.e. driver manager build options).

The constant is only documented as having an effect on Windows (where we actually do wide conversions). I think the simplest solution is to just nop it out on not-Windows:

diff --git a/ext/pdo_odbc/odbc_stmt.c b/ext/pdo_odbc/odbc_stmt.c
index 8786f2563e5..880b7b9732a 100644
--- a/ext/pdo_odbc/odbc_stmt.c
+++ b/ext/pdo_odbc/odbc_stmt.c
@@ -34,6 +34,7 @@ enum pdo_odbc_conv_result {
 
 static int pdo_odbc_sqltype_is_unicode(pdo_odbc_stmt *S, SQLSMALLINT sqltype)
 {
+#ifdef PHP_WIN32
 	if (!S->assume_utf8) return 0;
 	switch (sqltype) {
 #ifdef SQL_WCHAR
@@ -51,6 +52,9 @@ static int pdo_odbc_sqltype_is_unicode(pdo_odbc_stmt *S, SQLSMALLINT sqltype)
 		default:
 			return 0;
 	}
+#else
+	return 0;
+#endif
 }
 
 static int pdo_odbc_utf82ucs2(pdo_stmt_t *stmt, int is_unicode, const char *buf,

This also fixes the issue for me on SQL Server. (I couldn't reproduce it on Db2i, I guess it probably assumes an SQL_C_BINARY is UTF-8).

@lazerg
lazerg force-pushed the fix/gh-23444-pdo-odbc-utf8-conv branch from b169d2b to c6ed86d Compare August 25, 2026 18:10
@lazerg

lazerg commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

You're right, I've switched to your diff.

Two things back it up. With the attribute off, msodbcsql18 on unixODBC already round-trips CJK parameters, LIKE patterns and nvarchar columns correctly, so the conversion buys nothing there. And the manual documents the attribute as Windows only, and says the driver may do the conversion when it is false. Making it work everywhere would have been a new feature on a stable branch rather than a bug fix.

Your Db2i note decided it. My version sent UTF-16 as SQL_C_BINARY, so on a driver that reads SQL_C_BINARY as UTF-8 it would corrupt data that works today, and I can't test that platform.

One thing the no-op leaves behind, which I don't think belongs in this PR: on Windows a bound wide column gets a buffer of colsize+1 bytes while the driver writes two bytes per character, so a value longer than half the column width comes back truncated, plain ASCII included. An earlier revision of this PR hit that in the Windows CI job. I can send it separately if you want it.

I kept a regression test for the reported symptom. Without the patch it fails with HY090.

…Windows

The attribute is documented as Windows only, and the UTF-8 conversion it
relies on is compiled under #ifdef PHP_WIN32. Everywhere else
pdo_odbc_sqltype_is_unicode() still reported wide types as Unicode, so
parameters and columns were bound SQL_C_BINARY and then passed through
unconverted. Raw UTF-8 reached the server for an nvarchar parameter and
raw UTF-16 came back for an nvarchar column, and msodbcsql18 rejects a
parameter of odd byte length with HY090. Report it as not Unicode
outside Windows, which leaves the encoding to the driver as the default
already does.

On Windows the conversion runs but the data-at-exec branch declared the
unconverted byte length in SQL_LEN_DATA_AT_EXEC() while SQLPutData()
sent the converted bytes, so binding a non-ASCII parameter failed with
22026.

Closes phpGH-23444
@lazerg
lazerg force-pushed the fix/gh-23444-pdo-odbc-utf8-conv branch from c6ed86d to 53f3652 Compare August 25, 2026 18:53
@lazerg

lazerg commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

The Windows CI job caught something on top of your diff, so I've added one hunk.

With the no-op in place the test still failed on Windows with Driver 17, SQLSTATE 22026 at odbc_stmt.c:325. On Windows the conversion does run, but the data-at-exec branch puts the unconverted byte length in SQL_LEN_DATA_AT_EXEC() while SQLPutData() then sends the converted bytes. So binding any non-ASCII parameter under this attribute fails on Windows too, which the extension had no test for.

The hunk converts first and declares the converted length. Outside Windows pdo_odbc_utf82ucs2() returns CONV_NOT_REQUIRED, so that path is unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants