dbTalk Databases Forums  

Problem with Unicode Strings

comp.databases.oracle.misc comp.databases.oracle.misc


Discuss Problem with Unicode Strings in the comp.databases.oracle.misc forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
André Hartmann
 
Posts: n/a

Default Problem with Unicode Strings - 04-15-2008 , 09:04 AM






Hi,
I am having trouble with stuffing in an getting out unicode strings
into/from my database. I will briefly describe what I am doing, hoping that
someone can point me to a mistake and end my weeks of trying...

* Oracle 9i client and server, connecting via OCI from a C++ (MS Dev Studio
2005) application
* Client OS: German Windows XP, SP2
* Server: Oracle9.2.0.1 Enterprise on Windows XP SP2, NLS_LANGUAGE=AMERICAN
* Server character set (determined via "select value from
nls_database_parameters where parameter='NLS_CHARACTERSET';"): WE8MSWIN1252
* Client character set GERMAN_GERMANY.WE8MSWIN1252 (determined via "sqlplus
/nolog" and @.[%NLS_LANG%])

What I am doing in my program to stuff unicode in is this:

std::wstring strUnicode(L"Âb穾ü??déñf??");//not correctly displayed here
because this message is not unicode
....
wchar_t* pValue = (pWS_ == NULL) ? NULL : _wcsdup( strUnicode.c_str() );
size_t nSize = (pValue == NULL) ? 0 : (sizeof(wchar_t) * (1 +
wcslen(pValue)));
....
swdReturnCode = OCIBindByPos (
(OCIStmt*) m_pOCIStatement,
(OCIBind**) &pBindHandle,
(OCIError*) m_pConnection->_getOCIError(),
(ub4) columnIndex_,
(dvoid*) pValue,
(sb4) nSize,
(ub2) SQLT_STR,
(dvoid*) NULL,
(ub2*) NULL,
(ub2*) NULL,
(ub4) NULL,
(ub4*) NULL,
(ub4) OCI_DEFAULT
);
ub2 csid = OCI_UTF16ID;
swdReturnCode = OCIAttrSet(
(void *) pBindHandle
, (ub4) OCI_HTYPE_BIND
, (void *) &csid
, (ub4) 0
, (ub4) OCI_ATTR_CHARSET_ID
, m_pConnection->_getOCIError()
);
....

The string that I am stuffing in corresponds to the following sequence of
integers in my program's main memory:

194,98,231,169,190,252,1046,950,100,233,241,102,17 15,1492

When I select I get the following back (displayed as "Âb穾ü¿¿déñf¿¿"):

194,98,231,169,190,252,191,191,100,233,241,102,191 ,191

So it turns out that some characters have been transformed to be
upsode-down question marks. I see the same thing when I dont fetch with my
application but with the Oracle Enterprose Manager.

What is wrong here? Am I missing out some important conversion on the way?
Thanks in advance,
André






Reply With Quote
  #2  
Old   
Carlos
 
Posts: n/a

Default Re: Problem with Unicode Strings - 04-15-2008 , 11:13 AM






On Apr 15, 3:04 pm, "André Hartmann" <andrehartm... (AT) hotmail (DOT) com>
wrote:
Quote:
Hi,
I am having trouble with stuffing in an getting out unicode strings
into/from my database. I will briefly describe what I am doing, hoping that
someone can point me to a mistake and end my weeks of trying...

* Oracle 9i client and server, connecting via OCI from a C++ (MS Dev Studio
2005) application
* Client OS: German Windows XP, SP2
* Server: Oracle9.2.0.1 Enterprise on Windows XP SP2, NLS_LANGUAGE=AMERICAN
* Server character set (determined via "select value from
nls_database_parameters where parameter='NLS_CHARACTERSET';"): WE8MSWIN1252
* Client character set GERMAN_GERMANY.WE8MSWIN1252 (determined via "sqlplus
/nolog" and @.[%NLS_LANG%])

What I am doing in my program to stuff unicode in is this:

std::wstring strUnicode(L"Âb穾ü??déñf??");//not correctly displayed here
because this message is not unicode
...
wchar_t* pValue = (pWS_ == NULL) ? NULL : _wcsdup( strUnicode.c_str() );
size_t nSize = (pValue == NULL) ? 0 : (sizeof(wchar_t) * (1 +
wcslen(pValue)));
...
swdReturnCode = OCIBindByPos (
(OCIStmt*) m_pOCIStatement,
(OCIBind**) &pBindHandle,
(OCIError*) m_pConnection->_getOCIError(),
(ub4) columnIndex_,
(dvoid*) pValue,
(sb4) nSize,
(ub2) SQLT_STR,
(dvoid*) NULL,
(ub2*) NULL,
(ub2*) NULL,
(ub4) NULL,
(ub4*) NULL,
(ub4) OCI_DEFAULT
);
ub2 csid = OCI_UTF16ID;
swdReturnCode = OCIAttrSet(
(void *) pBindHandle
, (ub4) OCI_HTYPE_BIND
, (void *) &csid
, (ub4) 0
, (ub4) OCI_ATTR_CHARSET_ID
, m_pConnection->_getOCIError()
);
...

The string that I am stuffing in corresponds to the following sequence of
integers in my program's main memory:

194,98,231,169,190,252,1046,950,100,233,241,102,17 15,1492

When I select I get the following back (displayed as "Âb穾ü¿¿déñf¿¿"):

194,98,231,169,190,252,191,191,100,233,241,102,191 ,191

So it turns out that some characters have been transformed to be
upsode-down question marks. I see the same thing when I dont fetch with my
application but with the Oracle Enterprose Manager.

What is wrong here? Am I missing out some important conversion on the way?
Thanks in advance,
André

I cannot see the point of storing UNICODE (UTF8? UTF16?) in a
WE8MSWIN1252 database... (you don't specify the NLSupport codepage for
NVARCHARS/NCHARS)

The DB codepage should be unicode (AL32UTF8).

Cheers.

Carlos.


Reply With Quote
  #3  
Old   
Carlos
 
Posts: n/a

Default Re: Problem with Unicode Strings - 04-15-2008 , 11:13 AM



On Apr 15, 3:04 pm, "André Hartmann" <andrehartm... (AT) hotmail (DOT) com>
wrote:
Quote:
Hi,
I am having trouble with stuffing in an getting out unicode strings
into/from my database. I will briefly describe what I am doing, hoping that
someone can point me to a mistake and end my weeks of trying...

* Oracle 9i client and server, connecting via OCI from a C++ (MS Dev Studio
2005) application
* Client OS: German Windows XP, SP2
* Server: Oracle9.2.0.1 Enterprise on Windows XP SP2, NLS_LANGUAGE=AMERICAN
* Server character set (determined via "select value from
nls_database_parameters where parameter='NLS_CHARACTERSET';"): WE8MSWIN1252
* Client character set GERMAN_GERMANY.WE8MSWIN1252 (determined via "sqlplus
/nolog" and @.[%NLS_LANG%])

What I am doing in my program to stuff unicode in is this:

std::wstring strUnicode(L"Âb穾ü??déñf??");//not correctly displayed here
because this message is not unicode
...
wchar_t* pValue = (pWS_ == NULL) ? NULL : _wcsdup( strUnicode.c_str() );
size_t nSize = (pValue == NULL) ? 0 : (sizeof(wchar_t) * (1 +
wcslen(pValue)));
...
swdReturnCode = OCIBindByPos (
(OCIStmt*) m_pOCIStatement,
(OCIBind**) &pBindHandle,
(OCIError*) m_pConnection->_getOCIError(),
(ub4) columnIndex_,
(dvoid*) pValue,
(sb4) nSize,
(ub2) SQLT_STR,
(dvoid*) NULL,
(ub2*) NULL,
(ub2*) NULL,
(ub4) NULL,
(ub4*) NULL,
(ub4) OCI_DEFAULT
);
ub2 csid = OCI_UTF16ID;
swdReturnCode = OCIAttrSet(
(void *) pBindHandle
, (ub4) OCI_HTYPE_BIND
, (void *) &csid
, (ub4) 0
, (ub4) OCI_ATTR_CHARSET_ID
, m_pConnection->_getOCIError()
);
...

The string that I am stuffing in corresponds to the following sequence of
integers in my program's main memory:

194,98,231,169,190,252,1046,950,100,233,241,102,17 15,1492

When I select I get the following back (displayed as "Âb穾ü¿¿déñf¿¿"):

194,98,231,169,190,252,191,191,100,233,241,102,191 ,191

So it turns out that some characters have been transformed to be
upsode-down question marks. I see the same thing when I dont fetch with my
application but with the Oracle Enterprose Manager.

What is wrong here? Am I missing out some important conversion on the way?
Thanks in advance,
André

I cannot see the point of storing UNICODE (UTF8? UTF16?) in a
WE8MSWIN1252 database... (you don't specify the NLSupport codepage for
NVARCHARS/NCHARS)

The DB codepage should be unicode (AL32UTF8).

Cheers.

Carlos.


Reply With Quote
  #4  
Old   
Carlos
 
Posts: n/a

Default Re: Problem with Unicode Strings - 04-15-2008 , 11:13 AM



On Apr 15, 3:04 pm, "André Hartmann" <andrehartm... (AT) hotmail (DOT) com>
wrote:
Quote:
Hi,
I am having trouble with stuffing in an getting out unicode strings
into/from my database. I will briefly describe what I am doing, hoping that
someone can point me to a mistake and end my weeks of trying...

* Oracle 9i client and server, connecting via OCI from a C++ (MS Dev Studio
2005) application
* Client OS: German Windows XP, SP2
* Server: Oracle9.2.0.1 Enterprise on Windows XP SP2, NLS_LANGUAGE=AMERICAN
* Server character set (determined via "select value from
nls_database_parameters where parameter='NLS_CHARACTERSET';"): WE8MSWIN1252
* Client character set GERMAN_GERMANY.WE8MSWIN1252 (determined via "sqlplus
/nolog" and @.[%NLS_LANG%])

What I am doing in my program to stuff unicode in is this:

std::wstring strUnicode(L"Âb穾ü??déñf??");//not correctly displayed here
because this message is not unicode
...
wchar_t* pValue = (pWS_ == NULL) ? NULL : _wcsdup( strUnicode.c_str() );
size_t nSize = (pValue == NULL) ? 0 : (sizeof(wchar_t) * (1 +
wcslen(pValue)));
...
swdReturnCode = OCIBindByPos (
(OCIStmt*) m_pOCIStatement,
(OCIBind**) &pBindHandle,
(OCIError*) m_pConnection->_getOCIError(),
(ub4) columnIndex_,
(dvoid*) pValue,
(sb4) nSize,
(ub2) SQLT_STR,
(dvoid*) NULL,
(ub2*) NULL,
(ub2*) NULL,
(ub4) NULL,
(ub4*) NULL,
(ub4) OCI_DEFAULT
);
ub2 csid = OCI_UTF16ID;
swdReturnCode = OCIAttrSet(
(void *) pBindHandle
, (ub4) OCI_HTYPE_BIND
, (void *) &csid
, (ub4) 0
, (ub4) OCI_ATTR_CHARSET_ID
, m_pConnection->_getOCIError()
);
...

The string that I am stuffing in corresponds to the following sequence of
integers in my program's main memory:

194,98,231,169,190,252,1046,950,100,233,241,102,17 15,1492

When I select I get the following back (displayed as "Âb穾ü¿¿déñf¿¿"):

194,98,231,169,190,252,191,191,100,233,241,102,191 ,191

So it turns out that some characters have been transformed to be
upsode-down question marks. I see the same thing when I dont fetch with my
application but with the Oracle Enterprose Manager.

What is wrong here? Am I missing out some important conversion on the way?
Thanks in advance,
André

I cannot see the point of storing UNICODE (UTF8? UTF16?) in a
WE8MSWIN1252 database... (you don't specify the NLSupport codepage for
NVARCHARS/NCHARS)

The DB codepage should be unicode (AL32UTF8).

Cheers.

Carlos.


Reply With Quote
  #5  
Old   
Carlos
 
Posts: n/a

Default Re: Problem with Unicode Strings - 04-15-2008 , 11:13 AM



On Apr 15, 3:04 pm, "André Hartmann" <andrehartm... (AT) hotmail (DOT) com>
wrote:
Quote:
Hi,
I am having trouble with stuffing in an getting out unicode strings
into/from my database. I will briefly describe what I am doing, hoping that
someone can point me to a mistake and end my weeks of trying...

* Oracle 9i client and server, connecting via OCI from a C++ (MS Dev Studio
2005) application
* Client OS: German Windows XP, SP2
* Server: Oracle9.2.0.1 Enterprise on Windows XP SP2, NLS_LANGUAGE=AMERICAN
* Server character set (determined via "select value from
nls_database_parameters where parameter='NLS_CHARACTERSET';"): WE8MSWIN1252
* Client character set GERMAN_GERMANY.WE8MSWIN1252 (determined via "sqlplus
/nolog" and @.[%NLS_LANG%])

What I am doing in my program to stuff unicode in is this:

std::wstring strUnicode(L"Âb穾ü??déñf??");//not correctly displayed here
because this message is not unicode
...
wchar_t* pValue = (pWS_ == NULL) ? NULL : _wcsdup( strUnicode.c_str() );
size_t nSize = (pValue == NULL) ? 0 : (sizeof(wchar_t) * (1 +
wcslen(pValue)));
...
swdReturnCode = OCIBindByPos (
(OCIStmt*) m_pOCIStatement,
(OCIBind**) &pBindHandle,
(OCIError*) m_pConnection->_getOCIError(),
(ub4) columnIndex_,
(dvoid*) pValue,
(sb4) nSize,
(ub2) SQLT_STR,
(dvoid*) NULL,
(ub2*) NULL,
(ub2*) NULL,
(ub4) NULL,
(ub4*) NULL,
(ub4) OCI_DEFAULT
);
ub2 csid = OCI_UTF16ID;
swdReturnCode = OCIAttrSet(
(void *) pBindHandle
, (ub4) OCI_HTYPE_BIND
, (void *) &csid
, (ub4) 0
, (ub4) OCI_ATTR_CHARSET_ID
, m_pConnection->_getOCIError()
);
...

The string that I am stuffing in corresponds to the following sequence of
integers in my program's main memory:

194,98,231,169,190,252,1046,950,100,233,241,102,17 15,1492

When I select I get the following back (displayed as "Âb穾ü¿¿déñf¿¿"):

194,98,231,169,190,252,191,191,100,233,241,102,191 ,191

So it turns out that some characters have been transformed to be
upsode-down question marks. I see the same thing when I dont fetch with my
application but with the Oracle Enterprose Manager.

What is wrong here? Am I missing out some important conversion on the way?
Thanks in advance,
André

I cannot see the point of storing UNICODE (UTF8? UTF16?) in a
WE8MSWIN1252 database... (you don't specify the NLSupport codepage for
NVARCHARS/NCHARS)

The DB codepage should be unicode (AL32UTF8).

Cheers.

Carlos.


Reply With Quote
  #6  
Old   
André Hartmann
 
Posts: n/a

Default Re: Problem with Unicode Strings - 04-15-2008 , 11:26 AM



"Carlos" <miotromailcarlos (AT) netscape (DOT) net> schrieb im Newsbeitrag
news:c03f11e2-8cd4-41ff-bba6-b0a0f6c3c7e7 (AT) m3g2000hsc (DOT) googlegroups.com...
On Apr 15, 3:04 pm, "André Hartmann" <andrehartm... (AT) hotmail (DOT) com>
wrote:
Quote:
I cannot see the point of storing UNICODE (UTF8? UTF16?) in a
WE8MSWIN1252 database... (you don't specify the NLSupport codepage for
NVARCHARS/NCHARS)

The DB codepage should be unicode (AL32UTF8).

Hi, if that is so (unicode cannot be stored into databases that do not
have a unicode character set) then why can I create tables with unicode
columns (NCHAR, NVARCHAR2, NCLOB) in such databases? Wouldnt it be more
appropriate then for Oracle to cast errors when trying so? To say it the
other way round, the sheer fact that it is possible to declare unicode
columns in the database implied to me that it is possible to store such
values. Am I wrong here?

André





Reply With Quote
  #7  
Old   
André Hartmann
 
Posts: n/a

Default Re: Problem with Unicode Strings - 04-15-2008 , 11:26 AM



"Carlos" <miotromailcarlos (AT) netscape (DOT) net> schrieb im Newsbeitrag
news:c03f11e2-8cd4-41ff-bba6-b0a0f6c3c7e7 (AT) m3g2000hsc (DOT) googlegroups.com...
On Apr 15, 3:04 pm, "André Hartmann" <andrehartm... (AT) hotmail (DOT) com>
wrote:
Quote:
I cannot see the point of storing UNICODE (UTF8? UTF16?) in a
WE8MSWIN1252 database... (you don't specify the NLSupport codepage for
NVARCHARS/NCHARS)

The DB codepage should be unicode (AL32UTF8).

Hi, if that is so (unicode cannot be stored into databases that do not
have a unicode character set) then why can I create tables with unicode
columns (NCHAR, NVARCHAR2, NCLOB) in such databases? Wouldnt it be more
appropriate then for Oracle to cast errors when trying so? To say it the
other way round, the sheer fact that it is possible to declare unicode
columns in the database implied to me that it is possible to store such
values. Am I wrong here?

André





Reply With Quote
  #8  
Old   
André Hartmann
 
Posts: n/a

Default Re: Problem with Unicode Strings - 04-15-2008 , 11:26 AM



"Carlos" <miotromailcarlos (AT) netscape (DOT) net> schrieb im Newsbeitrag
news:c03f11e2-8cd4-41ff-bba6-b0a0f6c3c7e7 (AT) m3g2000hsc (DOT) googlegroups.com...
On Apr 15, 3:04 pm, "André Hartmann" <andrehartm... (AT) hotmail (DOT) com>
wrote:
Quote:
I cannot see the point of storing UNICODE (UTF8? UTF16?) in a
WE8MSWIN1252 database... (you don't specify the NLSupport codepage for
NVARCHARS/NCHARS)

The DB codepage should be unicode (AL32UTF8).

Hi, if that is so (unicode cannot be stored into databases that do not
have a unicode character set) then why can I create tables with unicode
columns (NCHAR, NVARCHAR2, NCLOB) in such databases? Wouldnt it be more
appropriate then for Oracle to cast errors when trying so? To say it the
other way round, the sheer fact that it is possible to declare unicode
columns in the database implied to me that it is possible to store such
values. Am I wrong here?

André





Reply With Quote
  #9  
Old   
André Hartmann
 
Posts: n/a

Default Re: Problem with Unicode Strings - 04-15-2008 , 11:26 AM



"Carlos" <miotromailcarlos (AT) netscape (DOT) net> schrieb im Newsbeitrag
news:c03f11e2-8cd4-41ff-bba6-b0a0f6c3c7e7 (AT) m3g2000hsc (DOT) googlegroups.com...
On Apr 15, 3:04 pm, "André Hartmann" <andrehartm... (AT) hotmail (DOT) com>
wrote:
Quote:
I cannot see the point of storing UNICODE (UTF8? UTF16?) in a
WE8MSWIN1252 database... (you don't specify the NLSupport codepage for
NVARCHARS/NCHARS)

The DB codepage should be unicode (AL32UTF8).

Hi, if that is so (unicode cannot be stored into databases that do not
have a unicode character set) then why can I create tables with unicode
columns (NCHAR, NVARCHAR2, NCLOB) in such databases? Wouldnt it be more
appropriate then for Oracle to cast errors when trying so? To say it the
other way round, the sheer fact that it is possible to declare unicode
columns in the database implied to me that it is possible to store such
values. Am I wrong here?

André





Reply With Quote
  #10  
Old   
Carlos
 
Posts: n/a

Default Re: Problem with Unicode Strings - 04-16-2008 , 08:44 AM



On Apr 15, 5:26 pm, "André Hartmann" <andrehartm... (AT) hotmail (DOT) com>
wrote:
Quote:
"Carlos" <miotromailcar... (AT) netscape (DOT) net> schrieb im Newsbeitragnews:c03f11e2-8cd4-41ff-bba6-b0a0f6c3c7e7 (AT) m3g2000hsc (DOT) googlegroups.com...
On Apr 15, 3:04 pm, "André Hartmann" <andrehartm... (AT) hotmail (DOT) com
wrote:>I cannot see the point of storing UNICODE (UTF8? UTF16?) in a
WE8MSWIN1252 database... (you don't specify the NLSupport codepage for
NVARCHARS/NCHARS)

The DB codepage should be unicode (AL32UTF8).

Hi, if that is so (unicode cannot be stored into databases that do not
have a unicode character set) then why can I create tables with unicode
columns (NCHAR, NVARCHAR2, NCLOB) in such databases? Wouldnt it be more
appropriate then for Oracle to cast errors when trying so? To say it the
other way round, the sheer fact that it is possible to declare unicode
columns in the database implied to me that it is possible to store such
values. Am I wrong here?

André

If you are using NVARCHAR2/NCHAR columns you are probably using
AL16UTF16 for these ones.

Can you confirm?

Cheers.

Carlos.


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.