CSV files are a hassle. Those darn double quotes are always screwing things
up. It is much easier to work with tab delimited data. I usually just suck
the record into Excel and then save is as tab delimited text. Unfortunately
these files are huge, so I can't do that.

Does anyone have a routine for parsing the data into something useable that
they are willing to share?

Thanks in advance,
Jeff

how about taking them into access 1st, then chop them up

Rich

Jeffrey Kaufman wrote:

The customer has given me almost 900 megs of data all in CSV. The data is
sitting in a directory in a Linux box in Phoenix. I don't have a lot of
options for manipulating the data. I have to deal with it as it is.


"Richard Wilson" <rwilson (AT) lakeside-systems (DOT) com> wrote

I dont know how much data access can handle

if 900mb doesnt max out access you can import csv and turn right around and
export tab

Jeffrey Kaufman wrote:

JK, Wintegrate just loves them.... and will deal with the double
quotes...
File, Import, browse to it, suck it into a dummy.file...
Hopefully you can create the dictionary and do a 'proof' report for the
client.yourself before writing the conversion program.
...
Other products handle them too, but we've done many conversions with
Wintegrate

The advantage with CSV is no size restrictions, we use them all the
time

ood Luck,

Nick

Jeffrey Kaufman wrote:

Jeffrey Kaufman wrote:
How about this:

001 subroutine csv.parser(string0,rec)
002 * 11-20-00 asb
003 * parse comma separated value record watching out for quotes ("")
004 * attribute delimited rec is returned
005
006 string = string0
007 rec = ""
008 an = 1
009 loop
010 if string[1,1] ne '"' then
011 result = field(string,",",1)
012 string = string[col2()+1,999999]
013 end else
014 pos = index(string,'",',1)
015 if pos eq 0 then pos = len(string)
016 result = string[1,pos]
017 string = string[pos+2,999999]
018 convert '"' to '' in result
019 end
020 rec<an> = trim(result)
021 until string eq "" do
022 an +=1
023 repeat
024
025 return


/Scott Ballinger
Pareto Corporation
Edmonds WA USA
206 713 6006

Rich,

Thank you for your input, but we are not on the same page. I am working
remotely. The data in on a Linux box. The amount of time it would take to
bring the data here, translate it, and send it back is prohibitive.

What I am looking for is a fool proof routing for parsing CSV data. I need
to account for situation such as this:

"2,000.00" - this example has a comma within the quotes

"1-1/2"" OVATION" - this example does not

Jeff

"Richard Wilson" <rwilson (AT) lakeside-systems (DOT) com> wrote

Hi Jeffry,

1. Install OpenQM on the box in Phoenix

2. Create a VOC "F" entry pointing to the directory containing the
CSV's.

3. Write a quickie BASIC program to read the CSV's, create a recipient
QM file and save the items.

4. T-DUMP/T-LOAD the files or do an ACCOUNT-SAVE/ACCOUNT-RESTORE

Look at the CSVDQ(), DPARSECSV(), MATREADCSV and READCSV functions and
statements to sort out #2 above. All character based, simple so easy
via ssh and sftp ... and PICK-like to boot. <g>

If the CSV's have header rows you could even get fancy and have your
BASIC program build dictionary items from these ...

Hope this helps,
-Tom

Gee, thanks.

"Tom deL" <ted (AT) blackflute (DOT) com> wrote

Now this looks helpful. Let me give it a try.

Thanks,
Jeff

"Scott Ballinger" <scott.ballinger (AT) gmail (DOT) com> wrote