dbTalk Databases Forums  

migrating oracle to postgres

comp.databases.postgresql comp.databases.postgresql


Discuss migrating oracle to postgres in the comp.databases.postgresql forum.



Reply
 
Thread Tools Display Modes
  #51  
Old   
Don Y
 
Posts: n/a

Default Re: Is PostgreSQL good? - 07-04-2011 , 02:24 PM






Hi Mathew,

On 7/4/2011 12:12 PM, Matthew Woodcraft wrote:
Quote:
In article<iul48k$9v1$1 (AT) speranza (DOT) aioe.org>, Don Y<nowhere (AT) here (DOT) com> wrote:
Are those parameters effectively "hints of a different color"?
I.e., hints regarding the makeup of the *data* themselves
instead of a particular *query*?

As Laurenz has explained, they're not.

But as I understand it, the PostgreSQL developers who are opposed to
adding 'traditional' hints think that the kind of hint you're
describing here is worth considering.
Because that information could be exploited by "smarter"
incarnations of the planner?

I.e., their philosophy being to let the planner do the thinking
and give it whatever information might be *helpful* to facilitate
its coming to a "better plan" and then just invest (development)
effort in *making* that "smarter planner"?

Presumably, the planner wouldn't be *obligated* to do as those
"non-hint hints" would otherwise suggest?

--don

Reply With Quote
  #52  
Old   
Laurenz Albe
 
Posts: n/a

Default Re: Is PostgreSQL good? - 07-05-2011 , 01:58 AM






Don Y wrote:
[about query hints]

Quote:
Ah! *But*, it can only "worsen (or improve) performance" -- not
affect the "correctness" of the result?
Correct.

[about bad execution plans]

Quote:
So, you stumble over them when a particular query (etc.)
unexpectedly proves to be more expensive than you had
"expected"?
Usually, that's how it is. Unless you Do It Right and test with
a realistic set of data beforehand.

Quote:
And, presumably, hints only benefit *specific* dataset-query
combinations. I.e., change the query and the hint's value may
change?
Yes. A query hint is part of the query and refers to certain
parts of it.

Quote:
The DBA (or more likely the programmer), who has knowledge about
the data that is not evident from the database layout (like "no male
can ever have an abortion") can sometimes know more than the planner.

OK. That would be very application-specific (and dataset-specific)
knowledge, though. (?)
Right. That's what I am talking about.

Quote:
Not quite, because routine vacuuming is not done by the end user
who executes statements, but by an administrator. The end user will
not have the necessary permissions to vacuum all tables in the database.

Yes, understood. I'm thinking about *my* applications where there
is an agent interposed between the end user and the DBMS acting
as if the administrator. So, it can run some commands on behalf of
the user and *then* decide to VACUUM (having inherent knowledge
of the consequences of those commands).
That sounds like a really bad idea.

If the database user that performs SQL on behalf of the end user
has superuser privileges, that constitutes an unnecessary risk.
Security holes or bugs in your software can cause much more damage
that way.

Quote:
With non-automatic vacuum, the DBA can for example schedule a daily
VACUUM of all tables at 11 p.m. because he or she knows that nothing
much is going on at that time.

That assumes you have sufficient free store to let a day's worth
of "garbage" accumulate.
Of course.

Quote:
WAL is unimportant for you only then if data loss is not a
problem for you, and yesterday's backup is good enough.

In my case, the opposite is true -- the application needs to
avoid "stumbling", if at all possible. Going back to
yesterday's dataset would be very painful.
Then you need to learn about WAL and point-in-time recovery.

Yours,
Laurenz Albe

Reply With Quote
  #53  
Old   
Don Y
 
Posts: n/a

Default Re: Is PostgreSQL good? - 07-05-2011 , 01:29 PM



Hi Laurenz,

On 7/4/2011 11:58 PM, Laurenz Albe wrote:

Quote:
And, presumably, hints only benefit *specific* dataset-query
combinations. I.e., change the query and the hint's value may
change?

Yes. A query hint is part of the query and refers to certain
parts of it.
So, unless you have accurate, prior knowledge of the dataset's
actual complexion, you stand a chance of shooting yourself in the
foot *if* you hint (based on assumptions you make that may not
turn out to be true)

Quote:
Not quite, because routine vacuuming is not done by the end user
who executes statements, but by an administrator. The end user will
not have the necessary permissions to vacuum all tables in the database.

Yes, understood. I'm thinking about *my* applications where there
is an agent interposed between the end user and the DBMS acting
as if the administrator. So, it can run some commands on behalf of
the user and *then* decide to VACUUM (having inherent knowledge
of the consequences of those commands).

That sounds like a really bad idea.

If the database user that performs SQL on behalf of the end user
has superuser privileges, that constitutes an unnecessary risk.
Security holes or bugs in your software can cause much more damage
that way.
No choice. There is no "organic" DBA involved. So, even if I
let autovacuum handle that aspect, there will always be other
aspects that have to be "coded" (and "Just Work")

Quote:
With non-automatic vacuum, the DBA can for example schedule a daily
VACUUM of all tables at 11 p.m. because he or she knows that nothing
much is going on at that time.

That assumes you have sufficient free store to let a day's worth
of "garbage" accumulate.

Of course.
OK. I can't guarantee that so it is something I will have to
be aware of.

Quote:
WAL is unimportant for you only then if data loss is not a
problem for you, and yesterday's backup is good enough.

In my case, the opposite is true -- the application needs to
avoid "stumbling", if at all possible. Going back to
yesterday's dataset would be very painful.

Then you need to learn about WAL and point-in-time recovery.
<grin> I'll add it to the List of Stuff I Must Learn.

Thanks!
--don

Reply With Quote
  #54  
Old   
Laurenz Albe
 
Posts: n/a

Default Re: Is PostgreSQL good? - 07-06-2011 , 02:00 AM



Don Y wrote:
Quote:
Yes, understood. I'm thinking about *my* applications where there
is an agent interposed between the end user and the DBMS acting
as if the administrator. So, it can run some commands on behalf of
the user and *then* decide to VACUUM (having inherent knowledge
of the consequences of those commands).

That sounds like a really bad idea.

If the database user that performs SQL on behalf of the end user
has superuser privileges, that constitutes an unnecessary risk.
Security holes or bugs in your software can cause much more damage
that way.

No choice. There is no "organic" DBA involved. So, even if I
let autovacuum handle that aspect, there will always be other
aspects that have to be "coded" (and "Just Work")
You can automatize administrative tasks, that's a good thing,
but that does not necessitate that the end user gets served by a
superuser account. Those things should be done separately.

You won't get away without some sort of human DBA.
You must at least ascertain that backups complete successfully
and space doesn't run out. And you need somebody who is able to
restore a backup. So at least when things go wrong, you need a DBA.

Yours,
Laurenz Albe

Reply With Quote
  #55  
Old   
Don Y
 
Posts: n/a

Default Re: Is PostgreSQL good? - 07-06-2011 , 03:00 AM



Hi Laurenz,

On 7/6/2011 12:00 AM, Laurenz Albe wrote:
Quote:
Don Y wrote:
Yes, understood. I'm thinking about *my* applications where there
is an agent interposed between the end user and the DBMS acting
as if the administrator. So, it can run some commands on behalf of
the user and *then* decide to VACUUM (having inherent knowledge
of the consequences of those commands).

That sounds like a really bad idea.

If the database user that performs SQL on behalf of the end user
has superuser privileges, that constitutes an unnecessary risk.
Security holes or bugs in your software can cause much more damage
that way.

No choice. There is no "organic" DBA involved. So, even if I
let autovacuum handle that aspect, there will always be other
aspects that have to be "coded" (and "Just Work")

You can automatize administrative tasks, that's a good thing,
but that does not necessitate that the end user gets served by a
superuser account. Those things should be done separately.

You won't get away without some sort of human DBA.
You must at least ascertain that backups complete successfully
and space doesn't run out. And you need somebody who is able to
restore a backup. So at least when things go wrong, you need a DBA.
The intend is to deploy in *devices* -- not "servers". For a
(crude) parallel, imagine the "contacts", etc. in your cell phone
were being managed by PostgreSQL. I.e., a set of relations
defining people, their phone numbers, log of recent calls, etc.

[recall, I am using a cell phone solely as an analogy as the
usage patterns are similar -- largely "fixed"]

The sorts of relations are all predefined. A (cell phone) "user"
could opt to add a name to his "contact list". Or, delete one.
Or "edit" an existing contact. Or, purge the log of the least
recent calls. etc.

He might even want to add "another" contact list (named "business
contacts"), etc.

But, he's not creating any arbitrary relations. And, the queries
he'll run are predefined -- with "fill in the blank" parameters, etc.

I.e., if PostgreSQL can't keep the relations intact, then it has
fundamental bugs (in which case, why would *anyone* be using it?).

If PostgreSQL isn't "ready for prime time", in that regard, I
have taken pains to define all my relations in such a way that I
could fall back to cruder tools (e.g., dbm-style where I do most
of the "work") and, even, a sparse VM implementation (linking
relations with actual pointers).

I have a trial, "server based" application running "unattended"
that has performed well so far (despite abuses to the server
itself). And, one of the two projects in the works currently
takes that a step further (more traffic, users, etc.). So,
I've got "an out" for the other project if I discover PG can't
"keep a clean house" without someone babysitting it... :<

<shrug> We'll see. AFAIK, it hasn't been tried on this large
a scale previously (?) As I said, the things I'm expecting from
the DBMS are probably different than what most users/DBA's would
expect.

--don

Reply With Quote
  #56  
Old   
Laurenz Albe
 
Posts: n/a

Default Re: Is PostgreSQL good? - 07-07-2011 , 02:05 AM



Don Y wrote:
Quote:
The sorts of relations are all predefined. A (cell phone) "user"
could opt to add a name to his "contact list". Or, delete one.
Or "edit" an existing contact. Or, purge the log of the least
recent calls. etc.

He might even want to add "another" contact list (named "business
contacts"), etc.
This is pretty vague, but it sounds like nothing you need a
superuser account for.
Keep administrative stuff (VACUUM, backups) separate from the rest.

Quote:
But, he's not creating any arbitrary relations. And, the queries
he'll run are predefined -- with "fill in the blank" parameters, etc.
Yup, that's exactly where the user can use SQL injection to break
into your database.

Quote:
I.e., if PostgreSQL can't keep the relations intact, then it has
fundamental bugs (in which case, why would *anyone* be using it?).
You misunderstood me. PostgreSQL will keep its stuff consistent.
If somebody breaks into your database with a superuser account,
he or she can very consistently read and change everything in the
database, access the file system, and theoretically do anything with
your machine that the OS user has permissions to do.

Quote:
shrug> We'll see. AFAIK, it hasn't been tried on this large
a scale previously (?) As I said, the things I'm expecting from
the DBMS are probably different than what most users/DBA's would
expect.
I don't want to play Kassandra here, but most of the people who
want to use a software for something else than the intended use
become quite unhappy in the end.

Yours,
Laurenz Albe

Reply With Quote
  #57  
Old   
Don Y
 
Posts: n/a

Default Re: Is PostgreSQL good? - 07-07-2011 , 02:34 AM



Hi Laurenz,

On 7/7/2011 12:05 AM, Laurenz Albe wrote:
Quote:
Don Y wrote:
The sorts of relations are all predefined. A (cell phone) "user"
could opt to add a name to his "contact list". Or, delete one.
Or "edit" an existing contact. Or, purge the log of the least
recent calls. etc.

He might even want to add "another" contact list (named "business
contacts"), etc.

This is pretty vague, but it sounds like nothing you need a
superuser account for.
I'm not claiming to need a "privileged" account. Rather, there is
an agent (a piece of software) that sits between the "user" and
the DBMS that interacts with the DB on behalf of the user and
reports back to the user.

So, it can run a query -- and then manually vacuum, etc.

Resources are scarce/fixed so you can't just arbitrarily do
something and let the pieces fall where they may. Instead,
you have to interact with the DB in ways that are more
"economical" (in space/time/power/etc.).

Quote:
Keep administrative stuff (VACUUM, backups) separate from the rest.

But, he's not creating any arbitrary relations. And, the queries
he'll run are predefined -- with "fill in the blank" parameters, etc.

Yup, that's exactly where the user can use SQL injection to break
into your database.
That's why there's an agent in the middle to "sanitize" all
interactions with the DBMS. The interface to the DBMS is
never "raw".

Quote:
I.e., if PostgreSQL can't keep the relations intact, then it has
fundamental bugs (in which case, why would *anyone* be using it?).

You misunderstood me. PostgreSQL will keep its stuff consistent.
Then there won't be any problems. :>

Quote:
If somebody breaks into your database with a superuser account,
he or she can very consistently read and change everything in the
database, access the file system, and theoretically do anything with
your machine that the OS user has permissions to do.
Understood. But, the same vulnerabilities exist in any PostgreSQL
instance. Less so, here, because the device isn't sitting on a
network, "exposed".

Quote:
shrug> We'll see. AFAIK, it hasn't been tried on this large
a scale previously (?) As I said, the things I'm expecting from
the DBMS are probably different than what most users/DBA's would
expect.

I don't want to play Kassandra here, but most of the people who
want to use a software for something else than the intended use
become quite unhappy in the end.
I don't see that what we are doing is in any way different from
what a PostgreSQL user could expect from a DB. The differences
are that our "users" are applications -- not "organic beings".
And, that there is typically only a single "organic" user
associated with each PG instance.

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.