Hi All,

I'm trying to make an example of handling nvarchar types in OpenAPI. What Ihave is essentially...

# include <wchar.h>
typedef struct _NVARCHAR
{
II_UINT2 actual_length; /* in bytes */
wchar_t actual_stuff[200]; /*well its actually nvarchar(200), so hang me! its just an example */
} MY_NVARCHAR;

MY_NVARCHAR input_widechar, return_widechar;

wcscpy(input_widechar.actual_stuff,
L"This is a big Hello World...and a wide one too!");
input_widechar.actual_length = wcslen(input_widechar.actual_stuff) * 2; /* no of bytes... */

....

I can then insert the data string into a table and read it back as well. Itseems to work.

My question is, when I set the actual_length of the input_widechar, is there a better way than a straight wcslen() * 2;

It just doesn't look right to me.

Martin Bowes

Hi Martin,

One consideration is that the size of wchar is 4 on some machines such
as Solaris Sparc and Linux systems. Instead of
wcslen(input_widechar.actual_stuff) * 2;
use
wcslen(input_widechar.actual_stuff) * sizeof(wchar_t);

Hope this helps,
Dave


--
thoda04

First, OpenAPI does not work with wchar_t types since on some platforms
this may be a 4 byte integer. NVARCHAR data is always UTF16 and two
bytes per character.

Second, the actual length is in characters, not bytes. The
IIAPI_DESCRIPTOR length is in bytes and includes the size of the actual
length field. This is a bit confusing, but the descriptor deals with
storage space while the actual length is describing the character data.

The following is from the OpenAPI User's Guide:

IIAPI_NVCH_TYPE - Specifies a variable-length National Character Set
string. Each character in the string is represented by an unsigned
two-byte integer holding a UTF-16 encoded character. The maximum size in
bytes for nvarchar data is stored as the ds_length parameter in the
corresponding descriptor. The actual length in characters of the varchar
data is stored in the first two bytes of the data buffer, followed by
the nvarchar data.


--
thogo01

Hi Marty

A recently worked on a number of bugs that did this type of thing in
an ESQLC program. The test case used MACROS for the assignments:

#define UC_ASSIGN(l, r) wcscpy((l.text), (r)); (l).len =
wcslen((l).text)
#define UC_TERM(v) (v).text[(v).len] = L'\0'

nvarchar struct {
short len;
wchar_t text[GW155_COL_LEN];
} res_nvarchar

As you can see there is no "* 2" on the length because the .len field
represents the number of characters (as Gordy pointed out)

You should make sure you have the fix for Bugs 123442, 123912 and
123728. These all relate to the use of nchar/nvarchar variables in ESQLC
and Java.

Regards

Alex


--
hanal04

Hi All,

So your using wchar_t stuff?

Given thogo01's description that wchar_t may be 4 bytes on some hosts, I'd assumed we might be better off with a stucture:
nvarchar struct {
II_UINT2 len; /* Number of characters, not bytes */
II_UINT2 text[];
}

I counter checked my existing programs usage of wchar_t and found by doing hex dumps of the data that I was indeed loading 4 byte character items into my test table.

I was busily rewriting my test case accordingly using iconv to make some test data.

Getting confused now!

Marty



-----Original Message-----
From: Ingres Forums [mailto:info-ingres (AT) kettleriverconsulting (DOT) com]
Sent: 07 October 2010 09:23
To: info-ingres (AT) kettleriverconsulting (DOT) com
Subject: Re: [Info-Ingres] OpenAPI and nvarchar type


Hi Marty

A recently worked on a number of bugs that did this type of thing in
an ESQLC program. The test case used MACROS for the assignments:

#define UC_ASSIGN(l, r) wcscpy((l.text), (r)); (l).len =
wcslen((l).text)
#define UC_TERM(v) (v).text[(v).len] = L'\0'

nvarchar struct {
short len;
wchar_t text[GW155_COL_LEN];
} res_nvarchar

As you can see there is no "* 2" on the length because the .len field
represents the number of characters (as Gordy pointed out)

You should make sure you have the fix for Bugs 123442, 123912 and
123728. These all relate to the use of nchar/nvarchar variables in ESQLC
and Java.

Regards

Alex


--
hanal04


_______________________________________________
Info-Ingres mailing list
Info-Ingres (AT) kettleriverconsulting (DOT) com
http://ext-cando.kettleriverconsulti...fo/info-ingres

And do OpenAPI and ESQLC use the same parts of GCA?

I use your ESQLC code to insert the Hello World string and the hex dump is showing each wide char is two bytes as I would expect.

I use my OpenAPI equivalent...with your US_ASSIGN stuff and NVARCHAR struct and my hex dump is showing 4 bytes per wide char....and a truncation in the string.

typedef struct _NVARCHAR
{
II_UINT2 len; /* The number of characters, not bytes */
wchar_t text[200]; /* So its fixed, hang me its only a test */
} MY_NVARCHAR;
....
UC_ASSIGN(input_widechar,
L"This is a big Hello World...and a wide one too!");
/* So the input_widechar len is 47 and everyone is happy*/

....
setDescrParm.sd_descriptor[1].ds_dataType = IIAPI_NVCH_TYPE;
setDescrParm.sd_descriptor[1].ds_nullable = FALSE;
setDescrParm.sd_descriptor[1].ds_length = sizeof(MY_NVARCHAR); /* 804? I expected 802 */
setDescrParm.sd_descriptor[1].ds_precision = 0;
setDescrParm.sd_descriptor[1].ds_scale = 0;
setDescrParm.sd_descriptor[1].ds_columnType = IIAPI_COL_QPARM;
setDescrParm.sd_descriptor[1].ds_columnName = NULL;
....
putParmParm.pp_parmData[1].dv_null = FALSE;
putParmParm.pp_parmData[1].dv_length = sizeof(MY_NVARCHAR);
putParmParm.pp_parmData[1].dv_value = &input_widechar;

Marty

-----Original Message-----
From: Alex Hanshaw [mailto:Alex.Hanshaw (AT) ingres (DOT) com]
Sent: 07 October 2010 10:12
To: Martin Bowes
Subject: RE: [Info-Ingres] OpenAPI and nvarchar type

Hi Marty

The len is the number of characters in the nvarchar struct used.
The gca length describes the number of bytes. So when GCA transmits the nvarchar it transmits sizeof(i2) + (var.len * sizeof(wchar_t)) bytes.

There's some code in adf/adu which checks the size of wchat_t and provides portability across platforms that use different sizes.
You may find this useful if your application is going to be used on more than one platform.



Alex

-----Original Message-----
From: Martin Bowes [mailto:martin.bowes (AT) ctsu (DOT) ox.ac.uk]
Sent: 07 October 2010 10:01
To: Ingres and related product discussion forum
Cc: Alex Hanshaw
Subject: RE: [Info-Ingres] OpenAPI and nvarchar type

Hi All,

So your using wchar_t stuff?

Given thogo01's description that wchar_t may be 4 bytes on some hosts, I'd assumed we might be better off with a stucture:
nvarchar struct {
II_UINT2 len; /* Number of characters, not bytes */
II_UINT2 text[];
}

I counter checked my existing programs usage of wchar_t and found by doing hex dumps of the data that I was indeed loading 4 byte character items into my test table.

I was busily rewriting my test case accordingly using iconv to make some test data.

Getting confused now!

Marty



-----Original Message-----
From: Ingres Forums [mailto:info-ingres (AT) kettleriverconsulting (DOT) com]
Sent: 07 October 2010 09:23
To: info-ingres (AT) kettleriverconsulting (DOT) com
Subject: Re: [Info-Ingres] OpenAPI and nvarchar type


Hi Marty

A recently worked on a number of bugs that did this type of thing in
an ESQLC program. The test case used MACROS for the assignments:

#define UC_ASSIGN(l, r) wcscpy((l.text), (r)); (l).len =
wcslen((l).text)
#define UC_TERM(v) (v).text[(v).len] = L'\0'

nvarchar struct {
short len;
wchar_t text[GW155_COL_LEN];
} res_nvarchar

As you can see there is no "* 2" on the length because the .len field
represents the number of characters (as Gordy pointed out)

You should make sure you have the fix for Bugs 123442, 123912 and
123728. These all relate to the use of nchar/nvarchar variables in ESQLC
and Java.

I've attached the tests case for Bug 123442 (esqlc and java) as you
might
me able to make some use of the code as a template.

If you need any help with wchar_t usage or have any questions don't
hesitate
to ask. Chris Clark wrote the java scripts for the above bugs and I'm
sure
he'll be able to answer any questions if I can't.

Regards

Alex


--
hanal04
------------------------------------------------------------------------
hanal04's Profile:
http://community.ingres.com/forum/member.php?userid=443
View this thread:
http://community.ingres.com/forum/sh...ad.php?t=12616

_______________________________________________
Info-Ingres mailing list
Info-Ingres (AT) kettleriverconsulting (DOT) com
http://ext-cando.kettleriverconsulti...fo/info-ingres

Hi Marty,

OpenAPI expects 2 bytes per Unicode glyph (character image, if you like)
where as on UNIX/Linux each Unicode glyph is 4 bytes (sizeof(wchar_t)).
I guess ESQL/C does the conversion for you?

So when describing and sending NVARCHAR data to OpenAPI,
sd_descriptor.ds_length value is the byte length of the UTF-16 string).
When putting the data with IIapi_putParam() dv_length is the byte length
of the UTF-16 string + 2 bytes, the extra 2 bytes are used to store the
glyph/character length. The data you are sending needs to be placed in a
new buffer whose size is dv_length bytes. The first two bytes of this
buffer contain the character/glyph length of the string being sent, like
with sending a VARCHAR value, with the rest of the buffer holding your
UTF-16 text.

Converting from UTF-32 to UTF-16 can be done using the function
ConvertUTF32toUTF16() in
http://svn.php.net/viewvc/pecl/ingre...242493&view=co.
The code came from the Unicode Consortium and should be what you need.

regards

grant

On 07/10/10 12:18, Martin Bowes wrote:

--
Grant Croker, Ingres Corp
Ingres PHP and Ruby maintainer
http://planetingres.org
You are disoriented. Blackness swims toward you like a school of eels
who have just seen something that eels like a lot.
- The Hitchhiker's Guide To The Galaxy text adventure, published by Infocom.

Hi Alex,

Definitly ESQLC is doing something which OpenAPI is not....perhaps an internal conversion from UTF-32 to UTF-16?

To load the data using OpenAPI I have to move away from wchar_t and do...
#include <iconv.h>
#include <errno.h>

typedef struct _NVARCHAR
{
II_UINT2 len; /* The number of characters, not bytes */
char text[200 * 2];
} MY_NVARCHAR;

/* Stuff for iconv */
char text[200];
char *inp_ptr, *out_ptr;
iconv_t cd; /* Conversion descriptor */
char fromcode[32], tocode[32];
size_t inbytesleft, outbytesleft, retcode;

strcpy(fromcode, "ASCII");
strcpy(tocode, "UTF-16LE");
cd = iconv_open(tocode, fromcode);
....Sundry error checks...
strcpy(text, "This is a big Hello World...and a wide one too!");
input_widechar.len = strlen(text); /* No of wide characters, not bytes! */

inp_ptr = text;
inbytesleft = (size_t )strlen(text);

out_ptr = input_widechar.text;
outbytesleft = (size_t )(strlen(text) * 2); /* No of bytes */

retcode = iconv(cd,
&inp_ptr, &inbytesleft,
&out_ptr, &outbytesleft);
....More error checking...
iconv_close(cd);

I can now load the 'input_widechar' using the exact same code as before.

This time the hex dump of the loaded field is in agreement with what is achieved using Alex's ESQLC code.

Marty

-----Original Message-----
From: Alex Hanshaw [mailto:Alex.Hanshaw (AT) ingres (DOT) com]
Sent: 07 October 2010 10:12
To: Martin Bowes
Subject: RE: [Info-Ingres] OpenAPI and nvarchar type

Hi Marty

The len is the number of characters in the nvarchar struct used.
The gca length describes the number of bytes. So when GCA transmits the nvarchar it transmits sizeof(i2) + (var.len * sizeof(wchar_t)) bytes.

There's some code in adf/adu which checks the size of wchat_t and provides portability across platforms that use different sizes.
You may find this useful if your application is going to be used on more than one platform.



Alex

-----Original Message-----
From: Martin Bowes [mailto:martin.bowes (AT) ctsu (DOT) ox.ac.uk]
Sent: 07 October 2010 10:01
To: Ingres and related product discussion forum
Cc: Alex Hanshaw
Subject: RE: [Info-Ingres] OpenAPI and nvarchar type

Hi All,

So your using wchar_t stuff?

Given thogo01's description that wchar_t may be 4 bytes on some hosts, I'd assumed we might be better off with a stucture:
nvarchar struct {
II_UINT2 len; /* Number of characters, not bytes */
II_UINT2 text[];
}

I counter checked my existing programs usage of wchar_t and found by doing hex dumps of the data that I was indeed loading 4 byte character items into my test table.

I was busily rewriting my test case accordingly using iconv to make some test data.

Getting confused now!

Marty



-----Original Message-----
From: Ingres Forums [mailto:info-ingres (AT) kettleriverconsulting (DOT) com]
Sent: 07 October 2010 09:23
To: info-ingres (AT) kettleriverconsulting (DOT) com
Subject: Re: [Info-Ingres] OpenAPI and nvarchar type


Hi Marty

A recently worked on a number of bugs that did this type of thing in
an ESQLC program. The test case used MACROS for the assignments:

#define UC_ASSIGN(l, r) wcscpy((l.text), (r)); (l).len =
wcslen((l).text)
#define UC_TERM(v) (v).text[(v).len] = L'\0'

nvarchar struct {
short len;
wchar_t text[GW155_COL_LEN];
} res_nvarchar

As you can see there is no "* 2" on the length because the .len field
represents the number of characters (as Gordy pointed out)

You should make sure you have the fix for Bugs 123442, 123912 and
123728. These all relate to the use of nchar/nvarchar variables in ESQLC
and Java.

I've attached the tests case for Bug 123442 (esqlc and java) as you
might
me able to make some use of the code as a template.

If you need any help with wchar_t usage or have any questions don't
hesitate
to ask. Chris Clark wrote the java scripts for the above bugs and I'm
sure
he'll be able to answer any questions if I can't.

Regards

Alex


--
hanal04
------------------------------------------------------------------------
hanal04's Profile:
http://community.ingres.com/forum/member.php?userid=443
View this thread:
http://community.ingres.com/forum/sh...ad.php?t=12616

_______________________________________________
Info-Ingres mailing list
Info-Ingres (AT) kettleriverconsulting (DOT) com
http://ext-cando.kettleriverconsulti...fo/info-ingres