dbTalk Databases Forums  

I have a database of 33 million addresses bank account nos etc

comp.databases.informix comp.databases.informix


Discuss I have a database of 33 million addresses bank account nos etc in the comp.databases.informix forum.



Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old   
reagan5609@googlemail.com
 
Posts: n/a

Default I have a database of 33 million addresses bank account nos etc - 11-27-2007 , 08:29 PM






I need to get them into SQL format they are just honking big flatfiles

any recommendations?

Reply With Quote
  #2  
Old   
Excalibur
 
Posts: n/a

Default Re: I have a database of 33 million addresses bank account nos etc - 11-28-2007 , 12:09 AM






Hi
Switch into sensible MV database records with a couple of btree indices then
encrypt the lot before you get hung. SQL injection attacks are specifically
targeted so MV is your best bet as every jerk will be headed the SQL way. I
think the fines for not properly protecting bankdata start at around $US
500,000 check on the latest Card rules for further info. You will find
amongst other things that all data stores must have unbeatable video
monitoring for a start.
Peter McMurray
<reagan5609 (AT) googlemail (DOT) com> wrote

Quote:
I need to get them into SQL format they are just honking big flatfiles

any recommendations?



Reply With Quote
  #3  
Old   
Kevin Powick
 
Posts: n/a

Default Re: I have a database of 33 million addresses bank account nos etc - 11-28-2007 , 12:16 AM



On Nov 27, 8:29 pm, reagan5... (AT) googlemail (DOT) com wrote:
Quote:
I need to get them into SQL format they are just honking big flatfiles
Many of the major SQL databases can facilitate the "bulk loading" of
large amounts of data from flat files. Two I know off the top of my
head are:

The COPY command for PostgreSQL and the LOAD DATA INFILE syntax for
MySQL.

--
Kevin Powick


Reply With Quote
  #4  
Old   
Art S. Kagel
 
Posts: n/a

Default Re: I have a database of 33 million addresses bank account nos etc - 11-28-2007 , 06:41 AM



On Nov 27, 8:29 pm, reagan5... (AT) googlemail (DOT) com wrote:
Quote:
I need to get them into SQL format they are just honking big flatfiles

any recommendations?
I assume you have legitimate rights to the data!?!?!?!

That said, As Kevin said, most databases have a load facility that can
handle one or more flatfile load formats. Informix has dbload which
can handle fixed field ASCII and ASCII delimited formats. It also
sports the High Performance Loader (HPLoader) which by default can
handle ASCII delimited formats but for which you can write your own
import function in C so it can handle just about any format of input.
Finally, you can always write a C/C++ program or script in any
advanced language like Perl, AWK, Ruby, etc. to read the file and
either reformat it or in the case of Perl with DBD/DBI for Informix
(or whatever database) or Ruby on Rails with the Informix (or
whatever) module, or PHP with the Informix (or whatever) module write
directly to the database.

Art S. Kagel


Reply With Quote
Reply




Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2009, Jelsoft Enterprises Ltd.