dbTalk Databases Forums  

Split a dmp file before import

comp.databases.oracle.misc comp.databases.oracle.misc


Discuss Split a dmp file before import in the comp.databases.oracle.misc forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Sashi
 
Posts: n/a

Default Split a dmp file before import - 11-10-2009 , 06:22 AM






Hi all, I have a dmp file that contains about 9 million rows.
Is there a utility/technique that will split the file into two (or
more) so that I can run the import in two (or more)
stages?

Thanks,
Sashi

Reply With Quote
  #2  
Old   
gazzag
 
Posts: n/a

Default Re: Split a dmp file before import - 11-10-2009 , 09:02 AM






On 10 Nov, 12:22, Sashi <small... (AT) gmail (DOT) com> wrote:
Quote:
Hi all, I have a dmp file that contains about 9 million rows.
Is there a utility/technique that will split the file into two (or
more) so that I can run the import in two (or more)
stages?

Thanks,
Sashi
1. What Oracle version?
2. What operating system?

Importing from a compressed pipe springs to mind but that's subject to
your reply to my second question.

HTH

-g

Reply With Quote
  #3  
Old   
Mark D Powell
 
Posts: n/a

Default Re: Split a dmp file before import - 11-10-2009 , 09:39 AM



On Nov 10, 10:02*am, gazzag <gar... (AT) jamms (DOT) org> wrote:
Quote:
On 10 Nov, 12:22, Sashi <small... (AT) gmail (DOT) com> wrote:

Hi all, I have a dmp file that contains about 9 million rows.
Is there a utility/technique that will split the file into two (or
more) so that I can run the import in two (or more)
stages?

Thanks,
Sashi

1. What Oracle version?
2. What operating system?

Importing from a compressed pipe springs to mind but that's subject to
your reply to my second question.

HTH

-g
Sashi, I would say the short answer is NO. You can run table= imports
to bring only specified tables on a specific import and you can
separate bring indexes/constraints from the import of the table data
but you cannot split bringing in a specific table in the middle nor
run the import without access to the entire dmp file series.

You can use the query= feature to split the export of the data from
one table into separate dmp files so that it could in effect be
brought back in pieces.

Also you can export partitioned tables by partition and bring back
partitions depending on your Oracle version and utility choices.

But once you have a dmp file your options are limited based on how the
file was created and what it holds except that with the impdp utility
you can also use the query= parameter to filter the input stream. So
if you know your data then in theory you could potentially bring
partial table contents but you woul d still need the entire complete
dmp file (and the dmp file would have had to be produced by expdp)

Generally speaking 9 millions rows is not a lot. More important is
the total size in Megabytes, Gigabytes, or Terabytes of data that the
9 million rows represent.

HTH -- Mark D Powell --

Reply With Quote
  #4  
Old   
joel garry
 
Posts: n/a

Default Re: Split a dmp file before import - 11-10-2009 , 12:15 PM



On Nov 10, 4:22*am, Sashi <small... (AT) gmail (DOT) com> wrote:
Quote:
Hi all, I have a dmp file that contains about 9 million rows.
Is there a utility/technique that will split the file into two (or
more) so that I can run the import in two (or more)
stages?

Thanks,
Sashi
Mark's answer pretty much says it all, but there may be more info
available if you tell us what problem you are trying to solve. Disk
space? Speed of imp? Redo generation? Trying to parallelize?

jg
--
@home.com is bogus.
Yay Stu! (Stu and I were buddies years ago): http://www.campinglife.com/output.cfm?ID=2209609

Reply With Quote
  #5  
Old   
Sashi
 
Posts: n/a

Default Re: Split a dmp file before import - 11-10-2009 , 02:56 PM



On Nov 10, 1:15*pm, joel garry <joel-ga... (AT) home (DOT) com> wrote:
Quote:
On Nov 10, 4:22*am, Sashi <small... (AT) gmail (DOT) com> wrote:

Hi all, I have a dmp file that contains about 9 million rows.
Is there a utility/technique that will split the file into two (or
more) so that I can run the import in two (or more)
stages?

Thanks,
Sashi

Mark's answer pretty much says it all, but there may be more info
available if you tell us what problem you are trying to solve. *Disk
space? *Speed of imp? *Redo generation? *Trying to parallelize?

jg
--
@home.com is bogus.
Yay Stu! *(Stu and I were buddies years ago):http://www.campinglife.com/output.cfm?ID=2209609
Thanks for your replies, and that pretty much sums it up well for me.
My problem is that my archiver keeps getting filled up, and I'm
running short of disk space. It's unable to generate enough undo
tablespace to represent my transaction.
I googled around and took the approach of commiting regularly and
using a buffer size of 50 MB.
The DMP file is actually a single table, and is about 950 MB.
So on my import command I set commit=Y and buffer=50000000.
This is 10.2.0.4.0 on solaris 10.

Regards,
Sashi

Reply With Quote
  #6  
Old   
Mark D Powell
 
Posts: n/a

Default Re: Split a dmp file before import - 11-11-2009 , 03:51 PM



On Nov 10, 3:56*pm, Sashi <small... (AT) gmail (DOT) com> wrote:
Quote:
On Nov 10, 1:15*pm, joel garry <joel-ga... (AT) home (DOT) com> wrote:





On Nov 10, 4:22*am, Sashi <small... (AT) gmail (DOT) com> wrote:

Hi all, I have a dmp file that contains about 9 million rows.
Is there a utility/technique that will split the file into two (or
more) so that I can run the import in two (or more)
stages?

Thanks,
Sashi

Mark's answer pretty much says it all, but there may be more info
available if you tell us what problem you are trying to solve. *Disk
space? *Speed of imp? *Redo generation? *Trying to parallelize?

jg
--
@home.com is bogus.
Yay Stu! *(Stu and I were buddies years ago):http://www.campinglife.com/output.cfm?ID=2209609

Thanks for your replies, and that pretty much sums it up well for me.
My problem is that my archiver keeps getting filled up, and I'm
running short of disk space. It's unable to generate enough undo
tablespace to represent my transaction.
I googled around and took the approach of commiting regularly and
using a buffer size of 50 MB.
The DMP file is actually a single table, and is about 950 MB.
So on my import command I set commit=Y and buffer=50000000.
This is 10.2.0.4.0 on solaris 10.

Regards,
Sashi- Hide quoted text -

- Show quoted text -
How do you back up your archive logs? (rman, manually to tape, etc...)

If you use rman to back up the archive logs you may want to run a
backup and delete task before you start and perhaps again while you
are running.

If you just back up the archived redo logs to tape then delete them
then again you may want to schedule this task to run just prior to
your load.

It does seem like you may need to allocate more space to the archive
log directory file system as 950MB is not that much redo though if
your system normally only generates a couple hundred meagbytes per day
I can understand not having enough space available to handle a special
load. On the other hand if this load is going to be repeated or is
typical of future load then your current issue is warning that your
archive file directory file system is too small. You might also need
to double check some of you other file system allocations such as for
backups, trace files, etc ....

HTH -- Mark D Powell --

Reply With Quote
  #7  
Old   
joel garry
 
Posts: n/a

Default Re: Split a dmp file before import - 11-11-2009 , 05:26 PM



On Nov 11, 1:51*pm, Mark D Powell <Mark.Powe... (AT) hp (DOT) com> wrote:
Quote:
On Nov 10, 3:56*pm, Sashi <small... (AT) gmail (DOT) com> wrote:



On Nov 10, 1:15*pm, joel garry <joel-ga... (AT) home (DOT) com> wrote:

On Nov 10, 4:22*am, Sashi <small... (AT) gmail (DOT) com> wrote:

Hi all, I have a dmp file that contains about 9 million rows.
Is there a utility/technique that will split the file into two (or
more) so that I can run the import in two (or more)
stages?

Thanks,
Sashi

Mark's answer pretty much says it all, but there may be more info
available if you tell us what problem you are trying to solve. *Disk
space? *Speed of imp? *Redo generation? *Trying to parallelize?

jg
--
@home.com is bogus.
Yay Stu! *(Stu and I were buddies years ago):http://www.campinglife..com/output.cfm?ID=2209609

Thanks for your replies, and that pretty much sums it up well for me.
My problem is that my archiver keeps getting filled up, and I'm
running short of disk space. It's unable to generate enough undo
tablespace to represent my transaction.
I googled around and took the approach of commiting regularly and
using a buffer size of 50 MB.
The DMP file is actually a single table, and is about 950 MB.
So on my import command I set commit=Y and buffer=50000000.
This is 10.2.0.4.0 on solaris 10.

Regards,
Sashi- Hide quoted text -

- Show quoted text -

How do you back up your archive logs? (rman, manually to tape, etc...)

If you use rman to back up the archive logs you may want to run a
backup and delete task before you start and perhaps again while you
are running.

If you just back up the archived redo logs to tape then delete them
then again you may want to schedule this task to run just prior to
your load.

It does seem like you may need to allocate more space to the archive
log directory file system as 950MB is not that much redo though if
your system normally only generates a couple hundred meagbytes per day
I can understand not having enough space available to handle a special
load. *On the other hand if this load is going to be repeated or is
typical of future load then your current issue is warning that your
archive file directory file system is too small. *You might also need
to double check some of you other file system allocations such as for
backups, trace files, etc ....

HTH -- Mark D Powell --
Some good points here, but I think the 950MB dump could possibly
explode into much more, as exports don't include indexes, just the
commands to create indexes, and of course, the data may be more dense
than in database format, depending.

I consider simply turning off archivelog mode, doing the import,
turning it back on, then taking a backup when I'm confronted by such
limitations on a non-repetitive basis. Also, I believe in the past I
had situations where doing the index generation after the import
relieved a lot of the undo issue. Nowadays I just keep an undo more
than half as big as the db :-O (and maybe more, I can't recall offhand
if I was given the raid level I asked for on that device).

jg
--
@home.com is bogus.
"I think they're communists!" - Paul Reubens, in remanufactured Cheech
& Chong film.
http://lauren.vortex.com/archive/000627.html

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.