Postgres 8.4 and 9.0 have the parameter named
"effective_io_concurrency". The manual page is very short, it says the
following:

Sets the number of concurrent disk I/O operations that PostgreSQL
expects can be executed simultaneously. Raising this value will increase
the number of I/O operations that any individual PostgreSQL session
attempts to initiate in parallel. The allowed range is 1 to 1000, or
zero to disable issuance of asynchronous I/O requests.

http://www.postgresql.org/docs/curre...-resource.html

My initial understanding was that this was the size of the table,
containing aiocb pointers, so that PgSQL can launch up to 1000
simultaneous aio_read or aio_write, per process. While monitoring the
system, I noticed that there is no asynchronous I/O at all! Nothing,
nada, zilch! Then I noticed that the "postgres" binary, is not even
linked with libaio, so aio_read was out of the question:

-bash-3.2$ ldd postgres|grep libaio
-bash-3.2$

The platform is Postgres 9.0.1 on RH EL 5.5 x86-64. My understanding of
the "effective_io_concurrency" was apparently very wrong. What is the
"effective concurrency" and what are those "simultaneous I/O requests"
that man page is talking about. Can somebody please define in precise
terms what is it that this parameter defines? What kind of "concurrent
I/O" is Postgres doing without asynchronous I/O calls? If this parameter
is just a stub for the future reference, I'd like to know. Will Postgres
use asynchronous I/O? Is that planned?




--
http://mgogala.byethost5.com

On Fri, 08 Oct 2010 01:01:49 +0000, Mladen Gogala wrote:

The mystery deepens. I thought that this might be the size of the I/O
vector, for readv and writev routines, but not so. I did
"ltrace -e readv -p <PID> on a PID that was doing a large sequential
scan and not a single "readv" library call was encountered. All calls
were just plain and simple "read" calls. Where is the concurrency? I am
really curious now. The LWN article pompously announced that PostgreSQL
9.0 will use asynchronous I/O, with aio_read and aio_write. What does
effective_io_concurrency define? What kind of "concurrent I/O" is
Postgresql doing? This doesn't look very "concurrent":

read(65, "\16\0\0\0\210\254\333\240\1\0\4\0L\0P\0\0 \4
\0\0\0\0000\231\240\r\370\227p\2"..., 8192) = 8192
read(65, "\16\0\0\0000\1\334\240\1\0\4\0008\0 \1\0 \4
\0\0\0\0(\234\260\7`\231\220\5"..., 8192) = 8192
read(65, "\16\0\0\0\20;\334\240\1\0\4\0<\0(\1\0 \4
\0\0\0\0\360\233\36\10\320\232@\2"..., 8192) = 8192
read(65, "\16\0\0\0Pk\334\240\1\0\4\0004\0\300\0\0 \4 \0\0\0\0H\232p\v
\224P\f"..., 8192) = 8192
read(65, "\16\0\0\0\220\273C\241\1\0\4\0D\0p\0\0 \4
\0\0\0\0\230\234\320\6\220\233\16\2"..., 8192) = 8192
read(65, "\16\0\0\0P\311\335\240\1\0\4\0<\0008\1\0 \4
\0\0\0\0\240\231\300\fp\230`\2"..., 8192) = 8192
read(65, "\16\0\0\0\260*\335\240\1\0\4\0008\0\350\0\0 \4
\0\0\0\0\20\230\340\0178\224\256\7"..., 8192) = 8192
read(65, "\16\0\0\0\20\10\337\240\1\0\4\0004\0h\0\0 \4
\0\0\0\0\210\231\356\f\230\225\340\7"..., 8192) = 8192
read(65, "\16\0\0\0\220\310C\241\1\0\4\0@\0\260\0\0 \4
\0\0\0\0H\231p\r0\227.\4"..., 8192) = 8192
read(65, "\16\0\0\0\350-\301\241\1\0\4\0<\0X\0\0 \4
\0\0\0\0H\232p\v\0\226\216\10"..., 8192) = 8192

Descriptor 65 is a DB file:
[root@lpo-postgres-01 ~]# cd /proc/16663/fd
[root@lpo-postgres-01 fd]# ls -l 65
lrwx------ 1 postgres postgres 64 Oct 7 23:26 65 ->
/software/pgsql/m-over/PG_9.0_201008051/16417/1572186.7

So, essentially, the process is reading block by block, in a sequence.
What, exactly, does "effective_io_concurrency" mean?




--
http://mgogala.byethost5.com

Mladen Gogala wrote:
As I understand it, it's asking the kernel to prefetch data, using
posix_fadvise.


I see that you ask a fair number of technical questions on this
newsgroup, but this newsgroup doesn't have a large number of PostgreSQL
experts reading it.

Can I just check that you're aware that the PostgreSQL mailing lists
are much more lively, and the core PostgreSQL developers actively
participate? This question would probably suit pgsql-performance (if
you prefer to use a newsreader, it's
gmane.comp.db.postgresql.performance on news.gmane.org).

Also, searching the archives of pgsql-hackers
(gmane.comp.db.postgresql.devel.general) is often a good way to find
answers to this sort of question for yourself.

-M-

On Fri, 08 Oct 2010 17:43:08 +0100, Matthew Woodcraft wrote:

I posted it on pgsql-novice, in addition to this group. Bruce Momjian,
Robert Haas, Josh Berkus and Tom Lane frequent that group. Thanks for
letting me know about gmane, I subscribed instantly.



--
http://mgogala.byethost5.com