Hi to everybody,
i would like to find every record where the inserted text is not UTF-8,
it's possible?

The official MySQL manual doen'st report anything like this.
Thanks in advance.

Max

--
http://www.maxori.it
L'Arte è la capacità di fare cose di nessun valore
e di riuscire a venderle
F. Zappa

On 11/11/2010 8:03 AM, MacMax wrote:
How can you tell if the text is utf-8 or not?

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex (AT) attglobal (DOT) net
==================

Jerry Stuckle <jstucklex (AT) attglobal (DOT) net> wrote:

Ok, my problem is this:

this is an old DB, and is full of a lot of text (UTF8, ASCII, cp1252,
WIN1252, etc, etc), i would like to extract only the records NOT UTF8.
Now i use a PHP function to filter the results with mb_detect_encode()
but is too slow, because there are thousands of records.
I don't know if is possible with SQL.

Thanks

Max

P.S. sorry for my english
--
http://www.maxori.it
L'Arte è la capacità di fare cose di nessun valore
e di riuscire a venderle
F. Zappa

macmax (AT) nospam (DOT) it (MacMax) wrote:
OK.

There is no direct equivalent of mb_detect_encoding() in SQL.
I don't even believe something like that can be implemented in
a sensible way at all. The best you can get is a probability
that a string uses a certain encoding.

But if you know what *exactly* this function does, you might be
able to express it in SQL. There are some alternatives shown in
the comments here:

http://php.net/manual/en/function.mb...t-encoding.php

Still I doubt it would be any faster. The mb-* functions in PHP
are compiled C code. If those are slow, then a pure SQL
implementation will be even slower.


XL

On 11/11/2010 8:24 AM, MacMax wrote:
If you check, mb_detect_encode() is only a guess - an educated guess,
but still only a guess. It cannot and does not pretend to be an
absolute determination of the charset. Such code would be impossible.

--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex (AT) attglobal (DOT) net
==================

On Thu, 11 Nov 2010 14:24:41 +0100, MacMax wrote:
That's the fast way do it, unfortuantely. Your other alternative is
essentially extract each value as hex/binary, try to iconv it into UTF-8
and if iconv complains that it can't, then it's not UTF-8 and you can
flag it for review. The problem is that (looking at the actual bits)
ASCII is cp1252 with one bit always 0, UTF-8 is cp1252 with some
*combinations* of character sequences being disallowed, and it's very
very difficult for a program that is not written to the specific context
of your data to tell the difference.

--
I hate mornings. I know they hate me back, too.
-- Joel Gluth

Peter H. Coffin <hellsop (AT) ninehells (DOT) com> wrote:

Thanks to everybody.
My final solution is a self-made php function to convert any record in
html entities.
Any record have a check button to add himself into an array with ID and
charset, in accordance with the charset the function apply the right
conversion.
I've taked this decision afetr you advices.
Long, very long job, but now everything it's ok.

Max
--
http://www.maxori.it
L'Arte è la capacità di fare cose di nessun valore
e di riuscire a venderle
F. Zappa