Not sure if this is a bug or not, but it seems like one.

I am trying to load a data file into a table using the copy command.

When I do it on a small subset of the data (gotten using the Linux head command) it all works fine, but when I run it on the full dataset, I get the following error:
ERROR: repalloc: invalid request size 1073741824
WARNING: AbortTransaction and not in in-progress state
message type 0x5a arrived from server while idle
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.

The file I'm trying to load has 57,063,476 rows in it and the file is 8,786,592,005 bytes long.

Do I need to break the file I'm trying to load up, are there other options?

I'm running this on Redhat Shrike Linux 9 on an Intel 2.4GHz processor w/ 1.25G of RAM and I've got over 500GB of storage total, with over 130GB free.

Thanks,
Garrett Dangerfield.


---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to majordomo (AT) postgresql (DOT) org)

Garrett Dangerfield <dangerimp (AT) cox (DOT) net> writes:
This looks like a corrupt-data sort of problem. You could probably
identify a particular line of the COPY data as causing the failure,
if you were to break the source file down into smaller and smaller
segments. Please do that and show us the offending line (as well
as your table definition and PG version...)

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to majordomo (AT) postgresql (DOT) org