dbTalk Databases Forums  

Performance Bottleneck

comp.databases.postgresql.novice comp.databases.postgresql.novice


Discuss Performance Bottleneck in the comp.databases.postgresql.novice forum.



Reply
 
Thread Tools Display Modes
  #11  
Old   
michael@floog.net
 
Posts: n/a

Default Re: help with COPY from .csv file into database - 08-04-2004 , 12:21 PM






Ahhhh, I see.
The light is slowly beginning to filter through the cobwebs. :-)

Let's hope the first steps are the toughest and most confusing.

Mike

Quoting Oliver Elphick <olly (AT) lfix (DOT) co.uk>:

Quote:
COPY happens in a transaction; either all of the data goes in or none
of
it does.




---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo (AT) postgresql (DOT) org



Reply With Quote
  #12  
Old   
Oliver Elphick
 
Posts: n/a

Default Re: help with COPY from .csv file into database - 08-04-2004 , 12:22 PM






On Wed, 2004-08-04 at 17:16, michael (AT) floog (DOT) net wrote:
Quote:
Hi Oliver,

This time the copy command appeared to work, except for the "CFW" entry
in the integer-formatted column, but then trying to view the data shows
the long hyphenated line again and no readable data.

What do you think is causing this?
Precisely that. Either all the data goes in or none of it does.

Clean up your input file and it will work.

--
Oliver Elphick olly (AT) lfix (DOT) co.uk
Isle of Wight http://www.lfix.co.uk/oliver
GPG: 1024D/A54310EA 92C8 39E7 280E 3631 3F0E 1EC0 5664 7A2F A543 10EA
========================================
"And not only so, but we glory in tribulations also;
knowing that tribulation worketh patience; And
patience, experience; and experience, hope."
Romans 5:3,4


---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo (AT) postgresql (DOT) org



Reply With Quote
  #13  
Old   
michael@floog.net
 
Posts: n/a

Default Re: help with COPY from .csv file into database - 08-04-2004 , 02:15 PM



Definitely will do this when I get home from work tonight.
Right now I'm ssh'ing into my box at home, but don't have an X-windows
server installed on my Win-bloze machine here at work. It's too
difficult to edit the .csv file from a console with vim. I need
x-windows so I can run OpenOffice spreadsheet to help see the file
better.

I'm wondering If I may need to reconfigure the fields using CREATE
TABLE, and add NULL to each field. I know that the .csv file
definitely has blank cells scattered throughout that were never filled
in with text or integers. I wonder if subsequent cell data is getting
moved because COPY won't accept empty space between comma separated
values. Or is this not an issue.

Mike


Quoting Oliver Elphick <olly (AT) lfix (DOT) co.uk>:

Quote:
Precisely that. Either all the data goes in or none of it does.

Clean up your input file and it will work.




---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings



Reply With Quote
  #14  
Old   
Oliver Elphick
 
Posts: n/a

Default Re: help with COPY from .csv file into database - 08-04-2004 , 03:09 PM



On Wed, 2004-08-04 at 19:15, michael (AT) floog (DOT) net wrote:
Quote:
Definitely will do this when I get home from work tonight.
Right now I'm ssh'ing into my box at home, but don't have an X-windows
server installed on my Win-bloze machine here at work. It's too
difficult to edit the .csv file from a console with vim. I need
x-windows so I can run OpenOffice spreadsheet to help see the file
better.

I'm wondering If I may need to reconfigure the fields using CREATE
TABLE, and add NULL to each field. I know that the .csv file
definitely has blank cells scattered throughout that were never filled
in with text or integers. I wonder if subsequent cell data is getting
moved because COPY won't accept empty space between comma separated
values. Or is this not an issue.
COPY will accept empty space, but it will treat an empty space as an
empty string, which is not the same as NULL. If that is meant to go in
an integer or date field, it will be invalid.

Use \N to represent nulls in the input file (or use some other
combination and specify it when invoking COPY).

Quote:
Quoting Oliver Elphick <olly (AT) lfix (DOT) co.uk>:


Precisely that. Either all the data goes in or none of it does.

Clean up your input file and it will work.


--
Oliver Elphick olly (AT) lfix (DOT) co.uk
Isle of Wight http://www.lfix.co.uk/oliver
GPG: 1024D/A54310EA 92C8 39E7 280E 3631 3F0E 1EC0 5664 7A2F A543 10EA
========================================
"And not only so, but we glory in tribulations also;
knowing that tribulation worketh patience; And
patience, experience; and experience, hope."
Romans 5:3,4


---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings



Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.