dbTalk Databases Forums  

Re: Storage space lost during an UPDATE

comp.databases.postgresql.novice comp.databases.postgresql.novice


Discuss Re: Storage space lost during an UPDATE in the comp.databases.postgresql.novice forum.



Reply
 
Thread Tools Display Modes
  #11  
Old   
Oliver Fromme
 
Posts: n/a

Default Re: Extended query: prepared statements list? - 07-15-2004 , 03:45 PM







Hi Marc,

M. Bastin wrote:
Quote:
Yes and no. I'm looking at the subject from the frontend-backend
protocol 3.0 point of view which is separated in the Extended Query
language and the Simple Query language.

With the PREPARE and EXECUTE statements you're accessing the Extended
Query language through the Simple Query language, which requires more
CPU time since your EXECUTE query itself is going to be parsed as a
Simple Query first before PostgreSQL realizes it must execute a
prepared statement, while when you send the commands directly through
the Extended Query language you skip that parsing step.
Thanks for the explanation. I've only been using Postgres
via the psql monitor and client applications (most of them
written in Python, some in Perl), so I'm not familiar with
the underlying client-server protocol.

You are right that the EXECUTE statement still has to be
parsed. However, I think the pasrsing overhead is small,
because the EXECUTE command has a very simple structure
("EXECUTE <plan> <arguments>"), as opposed to, say, the
very complicated synopsis of a SELECT command. Apart
from that, the overhead of the query planner is probably
much bigger, so using PREPARE + EXECUTE is probably still
a great win, I think.

I just wonder ... I'm currently writing a client app (in
Python) which has to insert quite a lot of things. This
is for a network traffic accounting system. In particular,
the program reads accumulated accounting data from a file,
pre-processes it and creates appropriate INSERT statements
(up to several hundreds or even thousands per session).
I wonder if it will be worth to PREPARE those inserts and
the EXECUTE them. But I guess it won't make much of a
difference, because the INSERT statements are very simple
anyway.

Best regards
Oliver

--
Oliver Fromme, secnetix GmbH & Co KG, Oettingenstr. 2, 80538 München
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

With Perl you can manipulate text, interact with programs, talk over
networks, drive Web pages, perform arbitrary precision arithmetic,
and write programs that look like Snoopy swearing.

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings



Reply With Quote
  #12  
Old   
M. Bastin
 
Posts: n/a

Default Re: Extended query: prepared statements list? - 07-15-2004 , 04:07 PM






At 10:45 PM +0200 7/15/04, Oliver Fromme wrote:
Quote:
Hi Marc,

M. Bastin wrote:
Yes and no. I'm looking at the subject from the frontend-backend
protocol 3.0 point of view which is separated in the Extended Query
language and the Simple Query language.

With the PREPARE and EXECUTE statements you're accessing the Extended
Query language through the Simple Query language, which requires more
CPU time since your EXECUTE query itself is going to be parsed as a
Simple Query first before PostgreSQL realizes it must execute a
prepared statement, while when you send the commands directly through
the Extended Query language you skip that parsing step.

Thanks for the explanation. I've only been using Postgres
via the psql monitor and client applications (most of them
written in Python, some in Perl), so I'm not familiar with
the underlying client-server protocol.

You are right that the EXECUTE statement still has to be
parsed. However, I think the pasrsing overhead is small,
It's 16,000 vs 1,500 microseconds on my system, plus the time for the
client to receive and parse the data which is the same in both cases.

Quote:
because the EXECUTE command has a very simple structure
("EXECUTE <plan> <arguments>"), as opposed to, say, the
very complicated synopsis of a SELECT command. Apart
from that, the overhead of the query planner is probably
much bigger, so using PREPARE + EXECUTE is probably still
a great win, I think.
Certainly for complex queries, but not for simple ones like "SELECT *
FROM mytable WHERE numcolumn > $1;"
Parsing a simple SELECT like this seems to take about the same time
as parsing an EXECUTE query.

I don't think you'd win anything by replacing INSERT with EXECUTE either.

Quote:
I just wonder ... I'm currently writing a client app (in
Python) which has to insert quite a lot of things. This
is for a network traffic accounting system. In particular,
the program reads accumulated accounting data from a file,
pre-processes it and creates appropriate INSERT statements
(up to several hundreds or even thousands per session).
I wonder if it will be worth to PREPARE those inserts and
the EXECUTE them. But I guess it won't make much of a
difference, because the INSERT statements are very simple
anyway.
I'm not familiar with Python but if it allows you to do a "COPY
mytable FROM STDIN;" then I would strongly recommend you'd use that
instead of INSERT. You build your data in a file and then you COPY
that file into your table in one step. You can import millions of
records in a couple of minutes like that. (You need to drop your
indexes on that table first and recreate them afterwards.)

Best,

Marc

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html



Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.