dbTalk Databases Forums  

Importing a UTF-8 Log File

microsoft.public.sqlserver.server microsoft.public.sqlserver.server


Discuss Importing a UTF-8 Log File in the microsoft.public.sqlserver.server forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Rubens
 
Posts: n/a

Default Importing a UTF-8 Log File - 08-26-2009 , 10:48 AM






First off, I apologize if I am posting to the wrong group. Also, I am not
very familiar with work with different file source encoding, so please
excuse my ignorance.

I need to be able to import a log file into a SQL 2000 table that uses the
UTF-8 text encoding. However, I've been getting invalid characters (i.e.
for the French language accents) appearing in the table. I have been able
to import the data correctly if I first save the file to Unicode encoding,
but this will not really be an option because this file is going to be
automated and received several times a day.

So how do I import this file so that the data appears correctly? I'd like
to save it as a DTS package, and here is the process I've run through using
SSMS 2008:

1. Launch the Import Wizard.
2. Options that I have selected in the General tab of the Flat File Source:

Locale: English (United States)
Unicode: Deselected
Code page: 1252 (ANSI Latin I)
Format: Ragged right (this is how the file is)
Text qualifier: Cannot change, set to <none>
Header row delimeter:{CR}{LF}
Header rows to skip: 0
Column names in the first data row is deselected

3. In the Advanced tab:

ColumnDelimeter: {CR}{LF}
DataType: string [DT_STR]
OutputColumnWidth: 2000
TextQualified: True

When you look at the Preview window (as well as if you proceed with the
import), the invalid characters appear.

Thank-you,
Rubens

Reply With Quote
  #2  
Old   
John Bell
 
Posts: n/a

Default Re: Importing a UTF-8 Log File - 08-27-2009 , 03:49 PM






"Rubens" <rubensrose (AT) hotmail (DOT) com> wrote

Quote:
First off, I apologize if I am posting to the wrong group. Also, I am not
very familiar with work with different file source encoding, so please
excuse my ignorance.

I need to be able to import a log file into a SQL 2000 table that uses the
UTF-8 text encoding. However, I've been getting invalid characters (i.e.
for the French language accents) appearing in the table. I have been able
to import the data correctly if I first save the file to Unicode encoding,
but this will not really be an option because this file is going to be
automated and received several times a day.

So how do I import this file so that the data appears correctly? I'd like
to save it as a DTS package, and here is the process I've run through
using SSMS 2008:

1. Launch the Import Wizard.
2. Options that I have selected in the General tab of the Flat File
Source:

Locale: English (United States)
Unicode: Deselected
Code page: 1252 (ANSI Latin I)
Format: Ragged right (this is how the file is)
Text qualifier: Cannot change, set to <none
Header row delimeter:{CR}{LF}
Header rows to skip: 0
Column names in the first data row is deselected

3. In the Advanced tab:

ColumnDelimeter: {CR}{LF}
DataType: string [DT_STR]
OutputColumnWidth: 2000
TextQualified: True

When you look at the Preview window (as well as if you proceed with the
import), the invalid characters appear.

Thank-you,
Rubens
Hi Rubens

SQL Server 2000 uses UCS-2 for unicode data and does not support UTF-8
encoding as a codepage (65001) for unicode data

http://support.microsoft.com/kb/232580

http://msdn.microsoft.com/en-us/libr...4(SQL.80).aspx

Code page 65001 can not be specified when using BCP/BULK INSERT. You may
want treat the data as binary, but then you may not be able to search the
data etc.

John

Reply With Quote
  #3  
Old   
Rubens
 
Posts: n/a

Default Re: Importing a UTF-8 Log File - 08-31-2009 , 07:33 AM



Thank-you John, I will pass this information on and see if we can develop
another solution.

Thanks,
Rubens

"John Bell" <jbellnewsposts (AT) hotmail (DOT) com> wrote

Quote:
"Rubens" <rubensrose (AT) hotmail (DOT) com> wrote in message
news:E8B8E111-A661-4A9B-BAAB-F9114BBF2155 (AT) microsoft (DOT) com...
First off, I apologize if I am posting to the wrong group. Also, I am
not very familiar with work with different file source encoding, so
please excuse my ignorance.

I need to be able to import a log file into a SQL 2000 table that uses
the UTF-8 text encoding. However, I've been getting invalid characters
(i.e. for the French language accents) appearing in the table. I have
been able to import the data correctly if I first save the file to
Unicode encoding, but this will not really be an option because this file
is going to be automated and received several times a day.

So how do I import this file so that the data appears correctly? I'd
like to save it as a DTS package, and here is the process I've run
through using SSMS 2008:

1. Launch the Import Wizard.
2. Options that I have selected in the General tab of the Flat File
Source:

Locale: English (United States)
Unicode: Deselected
Code page: 1252 (ANSI Latin I)
Format: Ragged right (this is how the file is)
Text qualifier: Cannot change, set to <none
Header row delimeter:{CR}{LF}
Header rows to skip: 0
Column names in the first data row is deselected

3. In the Advanced tab:

ColumnDelimeter: {CR}{LF}
DataType: string [DT_STR]
OutputColumnWidth: 2000
TextQualified: True

When you look at the Preview window (as well as if you proceed with the
import), the invalid characters appear.

Thank-you,
Rubens
Hi Rubens

SQL Server 2000 uses UCS-2 for unicode data and does not support UTF-8
encoding as a codepage (65001) for unicode data

http://support.microsoft.com/kb/232580

http://msdn.microsoft.com/en-us/libr...4(SQL.80).aspx

Code page 65001 can not be specified when using BCP/BULK INSERT. You may
want treat the data as binary, but then you may not be able to search the
data etc.

John

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.