dbTalk Databases Forums  

Dynamically allocated new virtual shared memory segment repeatdly

comp.databases.informix comp.databases.informix


Discuss Dynamically allocated new virtual shared memory segment repeatdly in the comp.databases.informix forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
marcelo jaime
 
Posts: n/a

Default Dynamically allocated new virtual shared memory segment repeatdly - 07-22-2010 , 03:02 PM






Since a long time we have some troubles with Shared Memory segments assigned
by the engine.
When we run an onstat - g seg, this one shows that is always assigning new
segments and it seems that they never release them, because sometimes
appears the following error:
[ENOMEM][12]: out of available data space, check system memory
parameters (e.g. MAXMEM)
How can we know the process these segments are generating? How can we
release them, in case we could do it, without shut down the base?

Thanks in advance

Reply With Quote
  #2  
Old   
Art Kagel
 
Posts: n/a

Default Re: Dynamically allocated new virtual shared memory segment repeatdly - 07-22-2010 , 03:16 PM






You can release shared memory segments (except for the RESIDENT segment,
Message segments, and the first virtual segment) by running "onmode -F".
You don't state what version you are running, so I'll warn you that on some
IDS 7.3x/9.30 and earlier releases running onmode -F can sometimes crash the
engine especially if you run it often, so try it the first few times during
off times to make sure it is safe if you have an older version of Informix.

To prevent allocation of new memory resources, you have two choices:

- Expand the initial virtual segment size to include all of the extra
memory that's being allocated over time (SHMVIRTSIZE) or
- find the complex queries and sorts that are making the requests for
more memory and eliminate or simplify them.


Now, hitting the error 12 means that the OS would not give Informix any more
memory. That can be caused by several factors:

- The kernel parameter that controls how much memory a single process can
have is set too low. Solution: tune he kernel parameter (usually MAXMEM)
- The kernel parameter that controls how much shared memory a single
process can have or the number of shared memory segments a process can
attache to is set too low. Solution: tune the kernel (usually SHMMAX or
SHMNMN)
- You do not have enough physical memory plus swap to permit the engine
to allocate any more memory. Solution: buy and install more memory and/or
create more swap space.
- You are running a 32bit version of Informix and require access to more
than the 2-3 GB of memory that a 32bit process can address. Solution:
Upgrade to a 64bit version of Informix.


Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art (AT) iiug (DOT) org)

Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
organization with which I am associated either explicitly, implicitly, or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.



On Thu, Jul 22, 2010 at 4:02 PM, marcelo jaime <emetejota (AT) gmail (DOT) com> wrote:

Quote:
Since a long time we have some troubles with Shared Memory segments
assigned by the engine.
When we run an onstat - g seg, this one shows that is always assigning new
segments and it seems that they never release them, because sometimes
appears the following error:
[ENOMEM][12]: out of available data space, check system memory
parameters (e.g. MAXMEM)
How can we know the process these segments are generating? How can we
release them, in case we could do it, without shut down the base?

Thanks in advance

_______________________________________________
Informix-list mailing list
Informix-list (AT) iiug (DOT) org
http://www.iiug.org/mailman/listinfo/informix-list


Reply With Quote
  #3  
Old   
Art Kagel
 
Posts: n/a

Default Re: Dynamically allocated new virtual shared memory segment repeatdly - 07-23-2010 , 10:30 AM



The engine will not free a virtual segment once it has been created under
the assumption that if the space was needed once, it will be needed again.
Some of the reasons for the multiple segments can be:

- SHMADD is too small
- Multiple sessions needed space simultaneously where if sequentially
less would be needed
- The segments space became fragmented so that later queries couldn't get
enough contiguous memory and so allocated another segment.

Realistically, these all resolve back to SHMADD is too small. And of
course, their presence at all indicates - as I already said - that probably
you should increase SHMVIRTSIZE.

If you SET EXPLAIN ON in some or all of your applications you can see which
queries are sorting, creating temporary indexes, or creating Hash Tables and
which are performing GROUP BY operations. These are the operations, in
general, which will require additional memory to complete.

There's no command that will tell you instantly which sessions allocated
memory. You could query the SMI tables about sessions and threads and
gather some of this to narrow down the search, but that's so time consuming
that normally the session that caused the problem is gone before you can
drill down that far.

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art (AT) iiug (DOT) org)

Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
organization with which I am associated either explicitly, implicitly, or by
inference. Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.



On Fri, Jul 23, 2010 at 11:18 AM, marcelo jaime <emetejota (AT) gmail (DOT) com> wrote:

Quote:
Dear ART,

Thank you for your answer.

My Informix version is 10.00 UC6 (i know 9/2010 end of support...) On Linux
Box (RHL).

I would like to know, besides complex queries and sort, which other actions
could generate the increase on these segments? Views?
Is there any tool, besides Set Explain On (4GL), Trace on (SPL) that allows
me to see why these segments keep on increasing and why these virtual
memory segments are not liberated?.

Thank you,

Marcelo


---------- Forwarded message ----------
From: Art Kagel <art.kagel (AT) gmail (DOT) com
Date: 2010/7/22
Subject: Re: Dynamically allocated new virtual shared memory segment
repeatdly
To: marcelo jaime <emetejota (AT) gmail (DOT) com
Cc: informix-list (AT) iiug (DOT) org


You can release shared memory segments (except for the RESIDENT segment,
Message segments, and the first virtual segment) by running "onmode -F".
You don't state what version you are running, so I'll warn you that on some
IDS 7.3x/9.30 and earlier releases running onmode -F can sometimes crash the
engine especially if you run it often, so try it the first few times during
off times to make sure it is safe if you have an older version of Informix.

To prevent allocation of new memory resources, you have two choices:

- Expand the initial virtual segment size to include all of the extra
memory that's being allocated over time (SHMVIRTSIZE) or
- find the complex queries and sorts that are making the requests for
more memory and eliminate or simplify them.


Now, hitting the error 12 means that the OS would not give Informix any
more memory. That can be caused by several factors:

- The kernel parameter that controls how much memory a single process
can have is set too low. Solution: tune he kernel parameter (usually
MAXMEM)
- The kernel parameter that controls how much shared memory a single
process can have or the number of shared memory segments a process can
attache to is set too low. Solution: tune the kernel (usually SHMMAX or
SHMNMN)
- You do not have enough physical memory plus swap to permit the engine
to allocate any more memory. Solution: buy and install more memory and/or
create more swap space.
- You are running a 32bit version of Informix and require access to
more than the 2-3 GB of memory that a 32bit process can address. Solution:
Upgrade to a 64bit version of Informix.


Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (art (AT) iiug (DOT) org)

Disclaimer: Please keep in mind that my own opinions are my own opinions
and do not reflect on my employer, Advanced DataTools, the IIUG, nor any
other organization with which I am associated either explicitly, implicitly,
or by inference. Neither do those opinions reflect those of other
individuals affiliated with any entity with which I am affiliated nor those
of the entities themselves.



On Thu, Jul 22, 2010 at 4:02 PM, marcelo jaime <emetejota (AT) gmail (DOT) com>wrote:

Since a long time we have some troubles with Shared Memory segments
assigned by the engine.
When we run an onstat - g seg, this one shows that is always assigning new
segments and it seems that they never release them, because sometimes
appears the following error:
[ENOMEM][12]: out of available data space, check system memory
parameters (e.g. MAXMEM)
How can we know the process these segments are generating? How can we
release them, in case we could do it, without shut down the base?

Thanks in advance

_______________________________________________
Informix-list mailing list
Informix-list (AT) iiug (DOT) org
http://www.iiug.org/mailman/listinfo/informix-list




_______________________________________________
Informix-list mailing list
Informix-list (AT) iiug (DOT) org
http://www.iiug.org/mailman/listinfo/informix-list


Reply With Quote
  #4  
Old   
Superboer
 
Posts: n/a

Default Re: Dynamically allocated new virtual shared memory segment repeatdly - 07-23-2010 , 10:51 AM



hmm long shot..

you could look at onstat -g ses and see who eats memory;
then you could do a onstat -g ses <sid>
in the past a huge ralloc was most of the time caused by not closeing/
freeing cursors/statements.
if i recall correctly you can do a onstat -g stm <sid>
which displays the sql for the session, maybe it is a pointer which
can be used to
kick a developer who forgets to close and free cursors/statements....


Superboer

On 23 jul, 17:30, Art Kagel <art.ka... (AT) gmail (DOT) com> wrote:
Quote:
The engine will not free a virtual segment once it has been created under
the assumption that if the space was needed once, it will be needed again..
Some of the reasons for the multiple segments can be:

* *- SHMADD is too small
* *- Multiple sessions needed space simultaneously where if sequentially
* *less would be needed
* *- The segments space became fragmented so that later queries couldn't get
* *enough contiguous memory and so allocated another segment.

Realistically, these all resolve back to SHMADD is too small. *And of
course, their presence at all indicates - as I already said - that probably
you should increase SHMVIRTSIZE.

If you SET EXPLAIN ON in some or all of your applications you can see which
queries are sorting, creating temporary indexes, or creating Hash Tables and
which are performing GROUP BY operations. *These are the operations, in
general, which will require additional memory to complete.

There's no command that will tell you instantly which sessions allocated
memory. *You could query the SMI tables about sessions and threads and
gather some of this to narrow down the search, but that's so time consuming
that normally the session that caused the problem is gone before you can
drill down that far.

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (a... (AT) iiug (DOT) org)

Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
organization with which I am associated either explicitly, implicitly, orby
inference. *Neither do those opinions reflect those of other individuals
affiliated with any entity with which I am affiliated nor those of the
entities themselves.

On Fri, Jul 23, 2010 at 11:18 AM, marcelo jaime <emetej... (AT) gmail (DOT) com> wrote:
Dear ART,

Thank you for your answer.

My Informix version is 10.00 UC6 (i know 9/2010 end of support...) On Linux
Box (RHL).

I would like to know, besides complex queries and sort, which other actions
could generate the increase on these segments? Views?
Is there any tool, besides Set Explain On (4GL), Trace on (SPL) that allows
me *to see why these segments keep on increasing and why these virtual
memory segments are not liberated?.

Thank you,

Marcelo

---------- Forwarded message ----------
From: Art Kagel <art.ka... (AT) gmail (DOT) com
Date: 2010/7/22
Subject: Re: Dynamically allocated new virtual shared memory segment
repeatdly
To: marcelo jaime <emetej... (AT) gmail (DOT) com
Cc: informix-l... (AT) iiug (DOT) org

You can release shared memory segments (except for the RESIDENT segment,
Message segments, and the first virtual segment) by running "onmode -F"..
You don't state what version you are running, so I'll warn you that on some
IDS 7.3x/9.30 and earlier releases running onmode -F can sometimes crash the
engine especially if you run it often, so try it the first few times during
off times to make sure it is safe if you have an older version of Informix.

To prevent allocation of new memory resources, you have two choices:

* *- Expand the initial virtual segment size to include all of the extra
* *memory that's being allocated over time (SHMVIRTSIZE) or
* *- find the complex queries and sorts that are making the requests for
* *more memory and eliminate or simplify them.

Now, hitting the error 12 means that the OS would not give Informix any
more memory. *That can be caused by several factors:

* *- The kernel parameter that controls how much memory a single process
* *can have is set too low. *Solution: tune he kernel parameter (usually
* *MAXMEM)
* *- The kernel parameter that controls how much shared memory a single
* *process can have or the number of shared memory segments a process can
* *attache to is set too low. *Solution: tune the kernel (usuallySHMMAX or
* *SHMNMN)
* *- You do not have enough physical memory plus swap to permit theengine
* *to allocate any more memory. *Solution: buy and install more memory and/or
* *create more swap space.
* *- You are running a 32bit version of Informix and require accessto
* *more than the 2-3 GB of memory that a 32bit process can address.*Solution:
* *Upgrade to a 64bit version of Informix.

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (a... (AT) iiug (DOT) org)

Disclaimer: Please keep in mind that my own opinions are my own opinions
and do not reflect on my employer, Advanced DataTools, the IIUG, nor any
other organization with which I am associated either explicitly, implicitly,
or by inference. *Neither do those opinions reflect those of other
individuals affiliated with any entity with which I am affiliated nor those
of the entities themselves.

On Thu, Jul 22, 2010 at 4:02 PM, marcelo jaime <emetej... (AT) gmail (DOT) com>wrote:

Since a long time we have some troubles with Shared Memory segments
assigned by the engine.
When we run an onstat - g seg, this one shows that is always assigningnew
segments and it seems that they never release them, because sometimes
appears the following error:
[ENOMEM][12]: out of available data space, check system memory
parameters (e.g. MAXMEM)
How can we know the process these segments are generating? How can we
release them, in case we could do it, without shut down the base?

Thanks in advance

_______________________________________________
Informix-list mailing list
Informix-l... (AT) iiug (DOT) org
http://www.iiug.org/mailman/listinfo/informix-list

_______________________________________________
Informix-list mailing list
Informix-l... (AT) iiug (DOT) org
http://www.iiug.org/mailman/listinfo/informix-list

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.