Hello,

I have a simple logging application that needs to be able to run unattended and reliably for as long as possible.

The memory that it uses is increasing as it runs.

Here is an simplified example that reproduces the problem.

int main()
{
while (true)
{
Connection c;
c.connect("database","localhost","user","pass");
Query q = c.query();
q << "INSERT INTO table_name (field_1 , field_2) VALUES ( 1 , 2 ) ;";
q.execute();
}
}

This memory usage of this program gradually rises over time, and has reached tens of megabytes.

It makes no difference if I move the connection out of the loop.

I don't think that it is a memory 'leak' as leak detectors can't find any leaks.

Is this normal ?

Alex

--
MySQL++ Mailing List
For list archives: http://lists.mysql.com/plusplus
To unsubscribe: http://lists.mysql.com/plusplus?unsu...ie.nctu.edu.tw

Alex Burton wrote:
According to what tool? The likes of Windows Task Manager and top(1) on
*ix are notoriously unreliable when it comes to detecting the true
memory usage of a process. They count things like OS-allocated buffers,
mapped DLLs (if a DLL is mapped by multiple processes, its memory
footprint is counted against all processes that use it), etc.

Do you mean the connection object, or both the object and the call to
its connect() method?

Unless your memory debugger is broken, there are no leaks, then.

Sorry to be cranky about this, but unless you can show me where in the
MySQL++ code the memory leak is, I don't know what else I can tell you.

--
MySQL++ Mailing List
For list archives: http://lists.mysql.com/plusplus
To unsubscribe: http://lists.mysql.com/plusplus?unsu...ie.nctu.edu.tw

If you are using Visual Studio, the problem is some kind of memory leak
having to do with the query object. I haven't narrowed it down, yet. I
hope I have some time to look into deeper.

Look at this thread for more info on it:
http://lists.mysql.com/plusplus/5743

Has anyone else had time to look into this memory leak? I barely have
time to read these emails, these days. =)

BTW, Warren, you are doing a great job.


Alex Burton wrote:
--
MySQL++ Mailing List
For list archives: http://lists.mysql.com/plusplus
To unsubscribe: http://lists.mysql.com/plusplus?unsu...ie.nctu.edu.tw

Thanks for your replies, Bill and Warren.

I can confirm that I am using Windows Task Manager and MS Visual C++
7.1.

Although I take the point about Windows Task manager Including OS
buffers etc, programs can be written with stable memory usage, and my
program using mysql is not one of them.

The problem appears not to be a 'leak' as in unreferenced memory, but
excessive usage.
Perhaps the mysql client api maintains it's own memory pool for some
performance reasons etc.

Thanks for the link to the archives Bill, I didn't search for memory
leaks, only for memory usage.

Alex

On 04/10/2006, at 12:44 PM, Bill K wrote:


--
MySQL++ Mailing List
For list archives: http://lists.mysql.com/plusplus
To unsubscribe: http://lists.mysql.com/plusplus?unsu...ie.nctu.edu.tw

Bill K wrote:
I read back through that thread, and saw why I forgot about this
issue...this post settled the issue for me:

http://lists.mysql.com/plusplus/5767

Do you not find that convincing?

Whether memory usage increases in this situation isn't in question.
Whose fault it is is the question. Matt seems to think it's the CRT,
and I don't see a reason to argue.

That said, I proposed a bunch of tests in that thread that didn't get
done. (Try different compilers, try v2.0.0, etc.) So, we're not
operating on complete data here.

--
MySQL++ Mailing List
For list archives: http://lists.mysql.com/plusplus
To unsubscribe: http://lists.mysql.com/plusplus?unsu...ie.nctu.edu.tw

I can tell you that it is mostly related to the C or C++ runtime, both
of them are known, in many compilers, for not freeing resources they
have made, for example mingw C++ runtime libstdc++ does not free
deallocated string memory until he wants, which can be a lot of time
after, what you can do is move the conecction, the query creation and
the inster query outside the loop, also try using a template query for
that, so you only need to give the changed arguments

Alex Burton wrote:

--
MySQL++ Mailing List
For list archives: http://lists.mysql.com/plusplus
To unsubscribe: http://lists.mysql.com/plusplus?unsu...ie.nctu.edu.tw

Warren Young wrote:
still exists for other developers to run into. It would be nice to know
of a work-around. I guess one (on the surface) is to recycle existing
query objects.
- Bill Krahmer

--
MySQL++ Mailing List
For list archives: http://lists.mysql.com/plusplus
To unsubscribe: http://lists.mysql.com/plusplus?unsu...ie.nctu.edu.tw

Bill K wrote:
I'm not entirely convinced of that, either. While MySQL++ proper may
not be leaking the memory, it may be possible to change it to avoid the
symptom. Until all those tests are tried, we don't know that, either.

But, this is not my itch: I don't use MySQL++ on Windows. At this
point, my limit is suggesting places to look for the workaround, or
things to try that will indirectly show where the workaround may be
found. Until someone finds that workaround, it'll be up to individual
MySQL++ users to come to their own accommodation with this CRT
feature/bug. That would be unfortunate, but I can live with it.

It's a similar situation with threading, only the thread question is
farther along: thread-safety also isn't my itch, but there's a document
stating what we have to do to be thread-safe w.r.t. the MySQL C library,
so I have a development plan to make MySQL++ take care of some of this
for you.

--
MySQL++ Mailing List
For list archives: http://lists.mysql.com/plusplus
To unsubscribe: http://lists.mysql.com/plusplus?unsu...ie.nctu.edu.tw

Warren, Bill

I have managed to modify the mysql++ code to remove the problem.

In the query ctors, instead of initialising the base class with std:stream(0) and then calling init(sbuffer_) in the body of the ctor, just initialise with std:stream(sbuffer_).

I havent looked through the visual c++ standard code enough to be able to explain it completely, but calling std:stream(0) and then calling init effectively calls basic_ostream::init twice. Initialising the stream twice must be what is causing the problem.

An alternative would be to default construct std:stream and then call init, but don't call std:stream(0) and then init.

I'm not sure what the meaning of calling std:stream(0) is anyway.

I am not sure what effect this change would have on mysql++ using other implementations of the c++ standard library, but from looking at the code in the gnu standard library, it looks like it will all work the same - except that it doesn't leak when you call init twice apparently.

Alex

On Thursday, October 05, 2006, at 05:12PM, Warren Young <mysqlpp (AT) etr-usa (DOT) com> wrote:

--
MySQL++ Mailing List
For list archives: http://lists.mysql.com/plusplus
To unsubscribe: http://lists.mysql.com/plusplus?unsu...ie.nctu.edu.tw

Warren, Bill

My memory usage is now completely stable - thanks for both your help.

Morals to the story.

Prefer RAII - if the basic_ostream didn't have an init method it couldn't have been mis used.

Don't attempt to stir in a bicycle when a recipe asks for a cup of sugar.
0 can in no way be considered a pointer to a stream buffer.

The basic_ostream doesn't appear to attempt to do anything with the stream buffer during the constructor, but it might so it would be good to make sure that the stream buffer were already constructed.

Anyway found the problem not entirely sure of the best fix.

Alex

On Friday, October 06, 2006, at 02:25AM, Alex Burton <alexibu (AT) mac (DOT) com> wrote:

--
MySQL++ Mailing List
For list archives: http://lists.mysql.com/plusplus
To unsubscribe: http://lists.mysql.com/plusplus?unsu...ie.nctu.edu.tw