Radicore Forum: Bug Reports » preg_replace operation distorts multibyte unicode characters

Home » RADICORE development » Bug Reports » preg_replace operation distorts multibyte unicode characters

Show: Today's Messages :: Polls :: Message Navigator
E-mail to friend

preg_replace operation distorts multibyte unicode characters [message #5293]

Wed, 13 January 2016 19:11

kong
Messages: 90
Registered: December 2011

Member

When a $where string includes multibyte characters such as Chinese, sometimes you will notice that the $where clause is not interpreted correctly.

This problem can traced back to the operation

Code: [Select all] [Show/ hide]

$where = trim(preg_replace("/\s+/", " ", $where));  // replace tabs and newlines with ' '

in function where2indexedArray ($where) in file include.library.inc.

Turns out that the /s criteria of the regex applies also to individual bytes of multibyte characters and when such a single byte meets /s criteria it would then be replaced with a "space byte", changing the multibyte character into something else. I solved this problem by adding the /u modifier to the regex:

Code: [Select all] [Show/ hide]

$where = trim(preg_replace("/\s+/u", " ", $where));  // replace tabs and newlines with ' '

For reference: http://www.regular-expressions.info/php.htmlQuote:

If you want your regex to treat Far East characters as individual characters, you'll either need to use the mb_ereg functions, or the preg functions with the /u modifier.

Report message to a moderator