dbTalk Databases Forums  

[BUGS] Patch for not going beyond NOFILE system limit

mailing.database.pgsql-bugs mailing.database.pgsql-bugs


Discuss [BUGS] Patch for not going beyond NOFILE system limit in the mailing.database.pgsql-bugs forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
Martin Pitt
 
Posts: n/a

Default [BUGS] Patch for not going beyond NOFILE system limit - 05-17-2004 , 07:34 AM






--yEPQxsgoJgBvi8ip
Content-Type: multipart/mixed; boundary="dDRMvlgZJXvWKvBx"
Content-Disposition: inline

--dDRMvlgZJXvWKvBx
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi PostgreSQL developers!

Jacek Drobiecki recently sent me a patch which stops postgresql to
actively violate the system limit of maximum open files
(RLIMIT_NOFILE) in src/backend/storage/file/fd.c, function
count_usable_fds().=20

This avoids irritating kernel logs (if system overstep violations are
enabled) and also the grsecurity alert when starting PostgreSQL.

Can you please adopt this patch?

Currently the modifications are only enabled when postgresql is
compiled with -DCHECK_RLIMIT_NOFILE. Of course you can also use it
unconditionally.

Thanks for considering and have a nice day!

Martin

--=20
Martin Pitt Debian GNU/Linux Developer
martin (AT) piware (DOT) de mpitt (AT) debian (DOT) org
http://www.piware.de http://www.debian.org

--dDRMvlgZJXvWKvBx
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=08check_rlimit_nofile
Content-Transfer-Encoding: quoted-printable

diff -ruN postgresql-7.4.2-old/src/backend/storage/file/fd.c postgresql-7.4=
..2/src/backend/storage/file/fd.c
--- postgresql-7.4.2-old/src/backend/storage/file/fd.c 2004-02-24 00:03:43.=
000000000 +0100
+++ postgresql-7.4.2/src/backend/storage/file/fd.c 2004-05-17 13:31:44.0000=
00000 +0200
@@ -50,6 +50,9 @@
#include "storage/fd.h"
#include "storage/ipc.h"
=20
+#ifdef CHECK_RLIMIT_NOFILE
+#include <sys/resource.h>
+#endif
=20
/* Filename components for OpenTemporaryFile */
#define PG_TEMP_FILES_DIR "pgsql_tmp"
@@ -272,15 +275,28 @@
int used =3D 0;
int highestfd =3D 0;
int j;
+#ifdef CHECK_RLIMIT_NOFILE
+ struct rlimit rlim;
+#endif
=20
size =3D 1024;
fd =3D (int *) palloc(size * sizeof(int));
=20
+#ifdef CHECK_RLIMIT_NOFILE
+ getrlimit(RLIMIT_NOFILE, &rlim);
+#endif
+
/* dup until failure ... */
for (;
{
int thisfd;
=20
+#ifdef CHECK_RLIMIT_NOFILE
+ /* Don't go beyond RLIMIT_NOFILE */
+ if (highestfd >=3D rlim.rlim_cur - 1)
+ break;
+#endif
+
thisfd =3D dup(0);
if (thisfd < 0)
{

--dDRMvlgZJXvWKvBx--

--yEPQxsgoJgBvi8ip
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFAqK79DecnbV4Fd/IRAmFpAKDxOUZpDztht73Yifc8fUfkVP2+lQCgv4Hd
k7FK218LTZ2qgIeQdQjRcsU=
=fiAI
-----END PGP SIGNATURE-----

--yEPQxsgoJgBvi8ip--

Reply With Quote
  #2  
Old   
Bruce Momjian
 
Posts: n/a

Default Re: [BUGS] Patch for not going beyond NOFILE system limit - 05-17-2004 , 07:55 AM







We have already fixed this in CVS and the fix will be in 7.5:

revision 1.107
date: 2004/02/23 20:45:59; author: tgl; state: Exp; lines: +145 -88
Do a direct probe during postmaster startup to determine the maximum
number of openable files and the number already opened. This eliminates
depending on sysconf(_SC_OPEN_MAX), and allows much saner behavior on
platforms where open-file slots are used up by semaphores.


---------------------------------------------------------------------------

Martin Pitt wrote:
-- Start of PGP signed section.
Quote:
Hi PostgreSQL developers!

Jacek Drobiecki recently sent me a patch which stops postgresql to
actively violate the system limit of maximum open files
(RLIMIT_NOFILE) in src/backend/storage/file/fd.c, function
count_usable_fds().

This avoids irritating kernel logs (if system overstep violations are
enabled) and also the grsecurity alert when starting PostgreSQL.

Can you please adopt this patch?

Currently the modifications are only enabled when postgresql is
compiled with -DCHECK_RLIMIT_NOFILE. Of course you can also use it
unconditionally.

Thanks for considering and have a nice day!

Martin

--
Martin Pitt Debian GNU/Linux Developer
martin (AT) piware (DOT) de mpitt (AT) debian (DOT) org
http://www.piware.de http://www.debian.org
[ Attachment, skipping... ]
-- End of PGP section, PGP failed!

--
Bruce Momjian | http://candle.pha.pa.us
pgman (AT) candle (DOT) pha.pa.us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster


Reply With Quote
  #3  
Old   
Tom Lane
 
Posts: n/a

Default Re: [BUGS] Patch for not going beyond NOFILE system limit - 05-18-2004 , 07:22 PM



Bruce Momjian <pgman (AT) candle (DOT) pha.pa.us> writes:
Quote:
We have already fixed this in CVS and the fix will be in 7.5:
I don't think that the previous changes address what the complainant
seems to really want, viz not even *try* to exceed RLIMIT_NOFILE.

However the patch certainly must be rejected since (a) it appears
unaware of the post-7.4 work in this area, (b) it has not been made
portable (eg, it will fail to compile on machines without getrlimit),
and (c) we do not like patches that add random manual configuration
symbols to the code ... especially undocumented ones.

If this issue seems important to you then please rework the patch to be
up-to-date and properly autoconfiscated, and resubmit.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings


Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.