Well, I figured out the problem, and was able to get the code to
successfully complete. It turns out a default transaction log file of
only 256MB was not large enough. I recreated it at 4096MB, and
everything work okay.


--
andrigtmiller

Ingres Forums wrote:

Very good.

Since you are comparing DBMSs you might find some use for my
presentation on Ingres tuning tools now you've got your application
working. http://www.rationalcommerce.com/papers/tuner.htm

--
Roy

UK Ingres User Association Conference 2011 will be on Tuesday June 7 2011.
Put the date in your diary today.

Ingres Forums wrote:


Is the reason that you 'needed' a larger transaction log file, that you
were attempting to run the operation as one transaction?

I assume that there is no business requirement to load all 1 million
Customer entities and 1 million PricingMethod entities in one or two
transactions.
Try to keep your transactions as small as the business limitations
allow.
If you 'commit' after groups of 100 or even 1000, then you would
probably not need a 4Gb transaction log file, but would probably have
completed with the default tx-log.

When copying data, it is tempting to just bung in the whole lot in one
go, but that is resource demanding.
Additionally, for every transaction log file, no matter what its size, I
can devise a data load that will cause the failure that you experienced.

Remember the method for eating chocolate elephants.
- A little bit at a time.

Michael

Also try doing the load after issuing the statement: set session with on_logfull=commit;

Martin Bowes

-----Original Message-----
From: Michael Dyer [mailto:Michael.Dyer (AT) ingres (DOT) com]
Sent: 15 September 2010 08:00
To: Ingres and related product discussion forum
Subject: Re: [Info-Ingres] Application hangs

Ingres Forums wrote:


Is the reason that you 'needed' a larger transaction log file, that you
were attempting to run the operation as one transaction?

I assume that there is no business requirement to load all 1 million
Customer entities and 1 million PricingMethod entities in one or two
transactions.
Try to keep your transactions as small as the business limitations
allow.
If you 'commit' after groups of 100 or even 1000, then you would
probably not need a 4Gb transaction log file, but would probably have
completed with the default tx-log.

When copying data, it is tempting to just bung in the whole lot in one
go, but that is resource demanding.
Additionally, for every transaction log file, no matter what its size, I
can devise a data load that will cause the failure that you experienced.

Remember the method for eating chocolate elephants.
- A little bit at a time.

Michael



_______________________________________________
Info-Ingres mailing list
Info-Ingres (AT) kettleriverconsulting (DOT) com
http://ext-cando.kettleriverconsulti...fo/info-ingres

Martin Bowes wrote:

Err, I guess that makes the program run, but COMMIT is supposed to mean
"I, as a human being with full knowledge of the meaning of the database,
*warrant* the database is now in a consistent state."

on_logfull=commit boils down to "Do random stuff to my database."

I'm not saying don't use it ever, but do feel very uneasy about it if
you ever do. Making the transaction log bigger is always Plan A.

--
Roy

UK Ingres User Association Conference 2011 will be on Tuesday June 7 2011.
Put the date in your diary today.

On Sep 15, 10:00*am, "Michael Dyer" <Michael.D... (AT) ingres (DOT) com> wrote:
Also what about the round trips to the database involved? Is a command
executed once for each row? I guess this could add to the overhead too

nikosv wrote:

[snip]

Colossal overhead. However in this case it seems unavoidable and
probably not important (the database needs to be loaded in this way only
once).

You are definitely right that a live application should avoid that sort
of thing. Applications that suck all the data into the client for
processing are between 30x to 50x slower than applications that send
the work to the DBMS server. Cursors are evil.

--
Roy

UK Ingres User Association Conference 2011 will be on Tuesday June 7 2011.
Put the date in your diary today.