dbTalk Databases Forums  

Slow writing data into BLOB through JDBC

comp.databases.informix comp.databases.informix


Discuss Slow writing data into BLOB through JDBC in the comp.databases.informix forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Georgi Cholakov
 
Posts: n/a

Default Slow writing data into BLOB through JDBC - 10-08-2004 , 02:20 AM






Hi, fellows

I have Informix 7.30.UC10, running on HP Unix. When I put data, larger
than 1MB into BYTE column using JDBC driver version 2.21.JC3 (JDK
1.4.2), it takes very long time to finish the operation. For example,
if the data is 1MB it takes 23 secs. When I try to put 18MB, the
duration of that operation is 440 secs and checkpoint duration in the
online.log is 10 secs - is it too long?

Did someone face this problem?
Any solutions?

Thank you for your help.

Georgi Cholakov
ISY Intellect
g.cholakov (AT) isy-dc (DOT) com

Reply With Quote
  #2  
Old   
John Carlson
 
Posts: n/a

Default Re: Slow writing data into BLOB through JDBC - 10-09-2004 , 11:02 PM






On 8 Oct 2004 00:20:55 -0700, g.cholakov (AT) isy-dc (DOT) com (Georgi Cholakov)
wrote:

Quote:
Hi, fellows

I have Informix 7.30.UC10, running on HP Unix. When I put data, larger
than 1MB into BYTE column using JDBC driver version 2.21.JC3 (JDK
1.4.2), it takes very long time to finish the operation. For example,
if the data is 1MB it takes 23 secs. When I try to put 18MB, the
duration of that operation is 440 secs and checkpoint duration in the
online.log is 10 secs - is it too long?

Did someone face this problem?
Any solutions?
Is the BLOB column stored in a regular dbspace or in a blobspace? Any
BLOB data stored in a regular dbspace will go through the buffers . .
..
JWC


Reply With Quote
  #3  
Old   
Georgi Cholakov
 
Posts: n/a

Default Re: Slow writing data into BLOB through JDBC - 10-11-2004 , 12:58 AM



Quote:
Is the BLOB column stored in a regular dbspace or in a blobspace? Any
BLOB data stored in a regular dbspace will go through the buffers . .
.
JWC
The BLOB is in dbspace. I will create a blobspace and make a test again.

Thank you for your help.

Georgi Cholakov


Reply With Quote
  #4  
Old   
Georgi Cholakov
 
Posts: n/a

Default Re: Slow writing data into BLOB through JDBC - 10-12-2004 , 12:56 AM



I created a blobspace and then created the table again, using the
following command:

CREATE TABLE BLOBTEST
(
ID INTEGER PRIMARY KEY,
BLOBCOL BYTE IN BLOBSPACE1
)

But the time needed to write data in this column (blobcol) is the same
as previous, when the blobcol was IN TABLE, not in BLOBSPACE1.

Am I missing something?

Thank you for your time.
Georgi Cholakov

Reply With Quote
  #5  
Old   
Ulf
 
Posts: n/a

Default Re: Slow writing data into BLOB through JDBC - 10-13-2004 , 01:06 PM



Hi !

The tests that I have done show the same result as you get. I thought
it was a problem using JDBC so I did the same thing with a 4GL program
and got the same result. It seems that storing large blobs takes a
long time. I also looked at other databases and found similar problems
noted for them.

If you use a smart blob space (version 9.X) and turn off the logging
it will be a lot faster, but you will sacrifice the benefits that
logging will give. The solution that we are using now is to store the
data in a file in the filesystem and then pick it up using a servlet
that then stores the file in a blobcolumn in the database. From the
user perspective it is very fast to store large objects, and it is
then stored in the database a few seconds later, transaction logging
is used in this case. Of course this method is not a 100 % safe, but
close enough for our purposes.


Regards


Ulf



g.cholakov (AT) isy-dc (DOT) com (Georgi Cholakov) wrote in message news:<be9f1956.0410112156.348fcecc (AT) posting (DOT) google.com>...
Quote:
I created a blobspace and then created the table again, using the
following command:

CREATE TABLE BLOBTEST
(
ID INTEGER PRIMARY KEY,
BLOBCOL BYTE IN BLOBSPACE1
)

But the time needed to write data in this column (blobcol) is the same
as previous, when the blobcol was IN TABLE, not in BLOBSPACE1.

Am I missing something?

Thank you for your time.
Georgi Cholakov

Reply With Quote
  #6  
Old   
Jon Ritson
 
Posts: n/a

Default Re: Slow writing data into BLOB through JDBC - 08-17-2011 , 09:32 AM



It would be interesting to know your blobpage size when using a blobspace ...

onstat -d for the blobspace should show it.

Problem with blobspaces is they are really optimized for "blobs of a similar size", so they are not really effective if you have a wide range of blobsizes (e.g. 50% are 2k in size and then 50% are 2 Mb in size).

If you have a requirement for "large blobs" which are all generally around 2 Mb then you could consider creating a blobspace with a 2 Mb blobpage size using "-g pagesize" argument to onspaces.

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.