Hi,

I work for a company in Canada where we have a need for French accented
characters in the D3 data. We are running D3 under SBS 2003(VM).

All of my users are connected to D3 via telnet from Windows machines. I
standardized on code page 437 (old DOS) and use the Alt-key combinations
to enter the data into D3. Display displays fine under telnet and
prints good now that I have set all the printers to CP437.

The big problem I am having is pulling data through ODBC and displaying
it in under Windows cause it does not use CP437 and the code page map is
completely different. On top of that that, the same data that I
transfered through ODBC or copied using TCL COPY command displays
perfectly in a DOS box under Windows cause it uses CP437.

I need to display and print French accented characters in the D3
environment(through telnet) and in Windows graphical programs. What do
you think is the best way to accomplish this?

Your opinions please,
TIA
Rick

Contact Raining Data Support in France:
support.fr (AT) ENLEVEZ-MOIrainingdata (DOT) com


Rick Smereka wrote:

Rick,

"Rick Smereka" <rick_smereka (AT) yahoo (DOT) com> wrote

I do know for sure that there's no way to translate OEM to ANSI codepages on
the fly in case of D3 Windows ODBC Server. The reason is that D3 ODBC Client
doesn't have any embedded translating facility. A proxy utility with
translation capabilities won't help either as far as ODBC protocol uses some
8-bit control codes.

Meanwhile D3 Linux since 7.2.0 version has embedded translation for ODBC
Server. Moreover, all D3 Linux distros are coming with predefined Russian
CP866 / CP1251 combination. Though it's absolutely undocumented, it will
work fine. Translation table is open for editing so that one can arrange it
for any national OEM/ANSI combination.

Besides D3 both jBASE and Cache' MV have IMHO more modern solution to this
problem. They both support Unicode, UTF8 in particular. jDP (ODBC-type)
connectivity for jBASE requires a lot of effort to work properly and seems
like still doesn't support any character translation. In it's turn Cache' MV
has probably the more seamless implementation of ODBC/OLE DB connectivity
with embedded UTF8/ANSI translation.

HTH
Nick

Nick,

Nikolai Lukin wrote:

Thanks for the info. This basically confirms what I had believed. I have
been thinking that one way to accomplish what I need is to create
attributes in the SQL file definition which call a BASIC subroutine that
do the conversion. As long as the user pulls from the correct attribute,
this should work. This is somewhat messy and error prone. Is there
another way that that I can accomplish this?

TIA,
Rick

Rick,

"Rick Smereka" <rick_smereka (AT) yahoo (DOT) com> wrote

This is one of the approaches, but it will work if none of extended ANSI
French chars interfere with Pick delimiters. Again you're right, it may well
open a can of worms.

Another possible ways are:

1. Use of Delphi-based tools or similar, which utilize BDE. BDE supports
translation on the fly among many codepages with virtually any ODBC driver.

2. If target RDBMS has a sort of command line client utility similar to
ISQL/SQLCMD of MS SQL, you can shell out queries from D3. I.e. EXECUTE
"!sqlcmd select * from mytable ..." CAPTURING ..., etc. SQLCMD will convert
OEM to ANSI on the fly.

These approaches are very limited, and I'm afraid, won't help.

3. The feasible one. Run D3 Linux on RedHat under VMWare Server for Windows,
which is now free. Use embedded char translation in D3 Linux ODBC Server.
Setup is quite tricky, but isn't invincible. Performance is as good as in
native Linux.

Should you need to have more details, please, don't hesitate to contact me
off CDP.

Cheers,
Nick

Nick,

Nikolai Lukin wrote:
Great list of options. Stuff I never considered and things I did not
know about.

Since I am using CP437 to enter and store the data in D3, it is well
away from MV delimiters (lowest ASCII code is 129, highest is 151).

The data will be pulled from D3 via ODBC using various apps, mostly Microsoft
Office and ilk. There will not necessarily be a target RDBMS.

The D3 option under VMWare running on RHL is very interesting and if I
was certain of D3 in our environment, I would seriously consider it.
Long-term, our parent company is going to impose their IT policy (Microsoft,
SQL Server/Access...yuck) on us.

Thanks for your help. It has been invaluable.

Rick

"Rick Smereka" <rick_smereka (AT) yahoo (DOT) com> wrote

Rick,

I this case you're supposed to use VBA. So that you may use oemtochar and
chartooem functions from user32.dll. In order to utilize them you just need
to declare these functions properly in a way like that:

Private Declare Function OemToChar Lib "user32" Alias "OemToCharA" (ByVal
lpszSrc As String, ByVal lpszDst As String) As Long
Private Declare Function CharToOem Lib "user32" Alias "CharToOemA" (ByVal
lpszSrc As String, ByVal lpszDst As String) As Long

Though I can't now guarantee for 100% that it will work for you, it did work
for me when I used to write some code in VBA/VB6. In VB.NET char translation
business is different, but I know my may around that. Let me know if you're
interested.

Nick

Nikolai Lukin wrote:
I believe there may be "issues" with running under VMWare, or any of
the other virtualization environments, as part of what your activation
code runs off is the NIC, and even a reboot of the VM may cause "havoc"
as the software emulation may provide a different/new identifier

You can also get the same sort of problems/issues if you are hooked up
to a SAN and go & move things about I believe

Hello Nick,
Nice to see You.

LCMapString is more useful. For convert MB string to WideChar (Unicode)
MultiByteToWideChar && WideCharToMultiByte API or StrConv(Str,
vbFromUnicode) in VB

LCMapString
The LCMapString function either maps an input character string to
another using a specified transformation or generates a sort key for
the input string.

int LCMapString(
LCID Locale, // locale identifier
DWORD dwMapFlags, // mapping transformation type
LPCTSTR lpSrcStr, // source string
int cchSrc, // number of characters in source string
LPTSTR lpDestStr, // destination buffer
int cchDest // size of destination buffer
);
Parameters
Locale
[in] Specifies a locale identifier. The locale provides a context for
the string mapping or sort key generation. An application can use the
MAKELCID macro to create a locale identifier.
dwMapFlags
[in] Specifies the type of transformation used during string mapping or
the type of sort key generated. An application can specify one or more
of the following options. Restrictions are noted following the table.
Option Meaning
LCMAP_BYTEREV Windows NT/2000/XP: Use byte reversal. For example, if
you pass in 0x3450 0x4822 the result is 0x5034 0x2248.
LCMAP_FULLWIDTH Uses wide characters (where applicable).
LCMAP_HALFWIDTH Uses narrow characters (where applicable).
LCMAP_HIRAGANA Hiragana.
LCMAP_KATAKANA Katakana.
LCMAP_LINGUISTIC_CASING Uses linguistic rules for casing, rather than
file system rules (the default). Valid with LCMAP_LOWERCASE or
LCMAP_UPPERCASE only.
LCMAP_LOWERCASE Uses lowercase.
LCMAP_SIMPLIFIED_CHINESE Windows NT 4.0 and later: Maps traditional
Chinese characters to simplified Chinese characters.
LCMAP_SORTKEY Produces a normalized wide character-sort key. For more
information, see lpDestStr and Remarks.
LCMAP_TRADITIONAL_CHINESE Windows NT 4.0 and later: Maps simplified
Chinese characters to traditional Chinese characters.
LCMAP_UPPERCASE Uses uppercase.


The following flags are used only with the LCMAP_SORTKEY flag. Flag
Meaning
NORM_IGNORECASE Ignores case.
NORM_IGNOREKANATYPE Does not differentiate between Hiragana and
Katakana characters. Corresponding Hiragana and Katakana will compare
as equal.
NORM_IGNORENONSPACE Ignores nonspacing. This flag also removes Japanese
accent characters.
NORM_IGNORESYMBOLS Ignores symbols.
NORM_IGNOREWIDTH Does not differentiate between a single-byte character
and the same character as a double-byte character.
SORT_STRINGSORT Treats punctuation the same as symbols.



If the LCMAP_SORTKEY flag is not specified, the LCMapString function
performs string mapping. In this case the following restrictions apply:


LCMAP_LOWERCASE and LCMAP_UPPERCASE are mutually exclusive.
LCMAP_HIRAGANA and LCMAP_KATAKANA are mutually exclusive.
LCMAP_HALFWIDTH and LCMAP_FULLWIDTH are mutually exclusive.
LCMAP_TRADITIONAL_CHINESE and LCMAP_SIMPLIFIED_CHINESE are mutually
exclusive.
LCMAP_LOWERCASE and LCMAP_UPPERCASE are not valid in combination with
any of these flags: LCMAP_HIRAGANA, LCMAP_KATAKANA, LCMAP_HALFWIDTH,
LCMAP_FULLWIDTH.
When the LCMAP_SORTKEY flag is specified, the LCMapString function
generates a sort key. In this case the following restriction applies:

LCMAP_SORTKEY is mutually exclusive with all other LCMAP_* flags, with
the sole exception of LCMAP_BYTEREV.
lpSrcStr
[in] Pointer to a source string that the function maps or uses for sort
key generation.
cchSrc
[in] Specifies the number of TCHARs in the string pointed to by the
lpSrcStr parameter.
This count can include the NULL terminator, or not include it. If the
NULL terminator is included in the character count, it does not greatly
affect the mapping behavior. That is because NULL is considered to be
unsortable, and always maps to itself.

A cchSrc value of -1 specifies that the string pointed to by lpSrcStr
is null-terminated. If this is the case, and LCMapString is being used
in its string-mapping mode, the function calculates the string's length
itself, and null-terminates the mapped string stored into *lpDestStr.

lpDestStr
[out] Pointer to a buffer that receives the mapped string or sort key.
If LCMAP_SORTKEY is specified, LCMapString stores a sort key into the
buffer. The sort key is stored as an array of byte values in the
following format:

[all Unicode sort weights] 0x01 [all Diacritic weights] 0x01 [all Case
weights] 0x01 [all Special weights] 0x00
Note that the sort key is null-terminated. This is true regardless of
the value of cchSrc. Also note that, even if some of the sort weights
are absent from the sort key, due to the presence of one or more ignore
flags in dwMapFlags, the 0x01 separators and the 0x00 terminator are
still present.

cchDest
[in] Specifies the size, in TCHARs, of the buffer pointed to by
lpDestStr.
If the function is being used for string mapping, the size is a
character count. If space for a NULL terminator is included in cchSrc,
then cchDest must also include space for a NULL terminator.

If the function is being used to generate a sort key, the size is a
byte count. This byte count must include space for the sort key 0x00
terminator.

If cchDest is zero, the function's return value is the number of
characters, or bytes if LCMAP_SORTKEY is specified, required to hold
the mapped string or sort key. In this case, the buffer pointed to by
lpDestStr is not used.

Return Values
If the function succeeds, and the value of cchDest is nonzero, the
return value is the number of characters, or bytes if LCMAP_SORTKEY is
specified, written to the buffer. This count includes room for a NULL
terminator.

If the function succeeds, and the value of cchDest is zero, the return
value is the size of the buffer in characters, or bytes if
LCMAP_SORTKEY is specified, required to receive the translated string
or sort key. This size includes room for a NULL terminator.

If the function fails, the return value is 0. To get extended error
information, call GetLastError. GetLastError may return one of the
following error codes:

ERROR_INSUFFICIENT_BUFFER
ERROR_INVALID_FLAGS
ERROR_INVALID_PARAMETER
Remarks
The mapped string is null terminated if the source string is null
terminated.

The ANSI version of this function maps strings to and from Unicode
based on the specified LCID's default ANSI code page.

For the ANSI version of this function, the LCMAP_UPPERCASE flag
produces the same result as CharUpper in the locale. Likewise, the
LCMAP_LOWERCASE flag produces the same result as CharLower. This
function always maps a single character to a single character. Note
that, in these cases, the function maps the lowercase I to the
uppercase I, even when the current language is Turkish or Azeri. To
change this for Turkish and Azeri, specify LCMAP_LINGUISTIC_CASING.

If LCMAP_UPPERCASE or LCMAP_LOWERCASE is set and if LCMAP_SORTKEY is
not set, the lpSrcStr and lpDestStr pointers can be the same.
Otherwise, the lpSrcStr and lpDestStr pointers must not be the same. If
they are the same, the function fails, and GetLastError returns
ERROR_INVALID_PARAMETER.

If the LCMAP_HIRAGANA flag is specified to map Katakana characters to
Hiragana characters, and LCMAP_FULLWIDTH is not specified, the function
only maps full-width characters to Hiragana. In this case, any
half-width Katakana characters are placed as-is in the output string,
with no mapping to Hiragana. An application must specify
LCMAP_FULLWIDTH if it wants half-width Katakana characters mapped to
Hiragana.

Even if the Unicode version of this function is called, the output
string is only in WCHAR or CHAR format if the string mapping mode of
LCMapString is used. If the sort key generation mode is used, specified
by LCMAP_SORTKEY, the output is an array of byte values. To compare
sort keys, use a byte-by-byte comparison.

An application can call the function with the NORM_IGNORENONSPACE and
NORM_IGNORESYMBOLS flags set, and all other options flags cleared, in
order to simply strip characters from the input string. If this is done
with an input string that is not null-terminated, it is possible for
LCMapString to return an empty string and not return an error.

The LCMapString function ignores the Arabic Kashida. If an application
calls the function to create a sort key for a string containing an
Arabic Kashida, there will be no sort key value for the Kashida.

The function treats the hyphen and apostrophe a bit differently than
other punctuation symbols, so that words like coop and co-op stay
together in a list. All punctuation symbols other than the hyphen and
apostrophe sort before the alphanumeric characters. An application can
change this behavior by setting the SORT_STRINGSORT flag. See
CompareString for a more detailed discussion of this issue.

When LCMapString is used to generate a sort key, by setting the
LCMAP_SORTKEY flag, the sort key stored into *lpDestStr may contain an
odd number of bytes. The LCMAP_BYTEREV option only reverses an even
number of bytes. If both options are chosen, the last (odd-positioned)
byte in the sort key is not reversed. If the terminating 0x00 byte is
an odd-positioned byte, then it remains the last byte in the sort key.
If the terminating 0x00 byte is an even-positioned byte, it exchanges
positions with the byte that precedes it.

When LCMAP_SORTKEY flag is specified, the function generates a sort key
that, when used in strcmp, produces the same order as when the original
string is used in CompareString. When LCMAP_SORTKEY flag is specified,
the output string is a string, but the character values are not
meaningful display values.

Windows 2000/XP: The ANSI version of this function will fail if it is
used with a Unicode-only locale. See Table of Language Identifiers.

Windows 95/98/Me: LCMapStringW is supported by the Microsoft Layer for
Unicode. To use this, you must add certain files to your application,
as outlined in Microsoft Layer for Unicode on Windows 95/98/Me Systems.

Requirements
Windows NT/2000/XP: Included in Windows NT 3.1 and later.
Windows 95/98/Me: Included in Windows 95 and later.
Header: Declared in Winnls.h; include Windows.h.
Library: Use Kernel32.lib.
Unicode: Implemented as Unicode and ANSI versions on Windows
NT/2000/XP. Also supported by Microsoft Layer for Unicode.

See Also

Nikolai Lukin wrote:

Nick,

I have VB6 here left from my predecessor. At this point, I am trying to
limit its use for various reasons.

I performed an experiment. I created a BASIC subroutine which converts
from CP437 to CP1252 and modified attribute 7 of a copied dictionary
item (last name attribute) to call this subroutine. When I produce a
report on the D3 side into a hold file and then copy that report to
Windows and view it in a Windows editor, the translation is successful.
When I pull the data into Excel using ODBC, the data does not display
correctly. I need to find a hex dump program so that I can look at the
file byte by byte and see if the data is being mangled by the ODBC
server or the data is in tact an Windows is refusing to display it
correctly. Also, when I import the D3 report into Excel (fixed field),
Excel asks for the 'file origin'. If I use Windows/ANSI (which I assume
is CP1252) or CP1252, the report displays correctly in Excel but this
again is from a D3 report simply copied.

BTW I am using the Wikipedia CP1252 web page as my reference.

I'll report back once I examine the files on the byte level

Rick