On May 3, 2011, at 8:00 AM, Roy Hann wrote:

[clip]

Oh ... this is so true. But not just regarding non-SQL DBMSes ...

Why is MySQL so successful? Because the vendors of industrial strength databases
have always and continue to sell to the C-level folks while pandering to the
Gartners and Forresters of the world.

Techies are ignored. Especially the "small market" techies.

And they refuse to let people shoot themselves in the foot. People don't like
the guards on their table saws. Everyone takes them off.

Let's take a few examples:

- ZERO major DBMSes come pre-installed with Linux Distros (although configured insecurely)
- ZERO major DBMSes can be installed using "yum" or "apt-get" without a single prompt
- ZERO major DBMSes expose the transaction audit stream in a sensible fashion (mostly out of the foolish fear of losing enterprise replication and mirroring licenses)
- ZERO major DBMSes take the PHP market seriously. CMSes are mostly written in PHP. The most popular ones are, for certain.
- ZERO major DBMSes make it dead simple to copy a database from one machine to another

All the major DBMS manufactures where so concerned about getting their lunch eaten by 3rd parties
at the enterprise level that they instead got their "rear ends" chewed off by the entry level.

BTW, MySQL really sucks. I am forced to work with it daily. I blame
its popularity on the major DBMS vendors. Their short-sightness and greed
forced people to use it. Now we are stuck with it.

Cheers,

Michael Leo
JSQL Product Manager
Kettle River Consulting

http://www.krc.mn
http://jsql.com

On 03/05/2011 14:00, Roy Hann wrote:
I think what you're saying is that Relational DBs are superb at
relationships, but sometimes that level of detail is not required for
the data that we want to retrieve. There is a tension between having to
be explicit about what information you want (every last join and
conditional match), so that you get it all, and just being able to ask
for the overall thing and get the rest as a matter of course.


Well a lot of developers actually deal with data as bundles by
serialising them for network transmission between client and server. The
data is the de-serialised on the remote client for saving on the
database, so in some sense the application developer is one step removed
from the relation complexity.

I don't think there is a silver bullet solution, but I would say that in
general there is a trend to make data less specific so that it's
retrival doesn't require absolute specification to find it. I would say
that the widespread use of tags is an example of this.


In the case of a newspaper, although the content is changing, it's not
at the same rate as in a banking application and there are a lot of
readers fetching the same data. Databases (of any kind) are slow, so a
broad query cache will save huge amounts of query time. By query cache,
I don't mean just a query plan, but also the results.

I implemented a simple version of this on an ingres system and
performance shot up. Before doing a query I compared query text with
queries that had previously been run. If the query hadn't been run
before, I continued to run the query and then save query text and
results in the cache. Next time around the query results come straight
from the cache without any database search. There would be good
performance uplift using this scheme for newspaper content because the
cache would only need invalidating (clearing out) occasionally to allow
changed content to be seen.

Developers can already do this using serialised content in blobs. The
downside is that managing inter- and intra-blob relationships is very
difficult and expensive in time and processing.

My caching example goes some of the way toward handling a complete "car"
as one, even though you have to say exactly what is in the car to the
finest detail.

There's no reason why more complex data models couldn't be manipulated
using a relational database using serialised data in blobs, though I
agree it would be sidestepping the natural way of using such a database.

Yes. Most complex types involve tree relationships and that's never been
a comfortable thing in SQL. The processing of XML trees using e4x can be
interesting (if slow).

I really think that like everything in life, there is a sweet spot for
different technologies and we'll never see one size fits all.

On net wrote:

That's getting close to what I am (trying) to say, but I think of it in
even simpler terms than that. I just want the DBMS to treat the values
that my application undestands as values.

I don't want it to take an interest in their internal structure and I
don't want it to make me take an interest. There is nothing in
relational theory that says that it should; in fact RT says values are
presumed to be atomic (i.e. have no internal structure) for the
purpose of relational manipulation. BLObs don't do it for me because
they're untyped. I definitely want types. But in the sense that BLObs
are atoms they're a step in the right direction.

Incidentally (and this will shock some of my friends) I'd quite like to
have an XML type in Ingres. I want it so that I never have to look
inside one of the bastard things ever again. Other people would
probably like it for different reasons.


--
Roy

UK Ingres User Association Conference 2011 will be on Tuesday June 7 2011.
Register at http://www.regonline.co.uk/ukiua2011

On Tue, 3 May 2011 17:26:36 +0000 (UTC)
Roy Hann <specially (AT) processed (DOT) almost.meat> wrote:

Be careful what you wish for, Roy. You surely remember what Codd
called "path independence". In RDMLSDB, he said,

"Since, in general, it is not practical to develop application
programs which test for all tree structurings permitted by the system,
[unmodified] programs fail when a change in structure becomes
necessary."

What happens to your applications when changes to the business require
changes to the XML structure? Are the XML documents really not
decomposable for any other purpose? Will there be no XPath in the DBMS
or in your application? If they cannot be sorted or decomposed, how
are they not just blobs? What elevates XML over PDF?

What verifies the correctness and consistency of your XML "atoms"? If
you say "the application", are you really prepared to rely on disparate
programmers, unaware over time and often distance of each others' work,
sharing no common understanding of the data, to mutually verify their
inputs according to one set of rules, lacking as they do any
technological support for logic? Do you think sprinkling the data with
XML pixie dust will upend *decades* of unhappy experience with
"application verified" data?

Car parts!? You cannot take refuge in metaphor, my good sir.
As far as I can tell, the thing you want has existed for a good long
while: the view. What do you seek from XML that a view doesn't do
already? If you say "update" I'm going to say "stored procedure", fair
warning!

If you want to push Ingres to advance the state of the art, rather than
chasing the XML fad already fading, ask for just one datatype: the
table. Nothing in the relational model says you can't have a table *in*
a column.

I don't know how SQL will need to be modified to deal with
table-columns. One option, "nothing", is what we'd get from XML
(unless we drag in XPath, in which case we then have worse than nothing:
*two* query languages). More likely we'd get row constructors, row
comparitors, and table comparitors; how else to restrict and sort by
the type? I don't know how we'd declare FK constraints on tables in
columns but, again, at least it's a *table*!

I can understand the vendors' motivation to add XML: they can sell it
to an audience ignorant of RM. It's no problem to them, because the
query engine need know nothing about it. What I don't understand,
really and truly, is your motivation. I hear you saying it would let
you deal with blocks of stuff that your application always treats as a
unit. In what way does XML suggest itself as a good way to organize
those blocks of stuff? How are you hoping that XML per se will make
your job easier? I can imagine lots of deficiencies, and I have in RM
an answer for any advantage you might propose. But I really do wonder
what's on your mind when you wish for XML.

--jkl

James K. Lowden wrote:

Good advice.

From a DBMS perspective, I don't care. It's absolutely none of my
concern.

Within the DBMS, yes, really. That is not to say that one can't have
suitable scalar functions on them, just as we do on strings and dates.

Maybe you are edging towards asking if it should be possible to have
expressions involving XML documents, like the way you can multiply
numbers or concatenate strings. That ability is not only not
absolutely required to have a perfectly usable DBMS, it is presently
implemented in a way that makes no sense. If I have two columns of
type "weight" it makes no sense to be allowed to multiply them
together, yet I am allowed to. SQL expressions are broken and I don't
really want to get hung up on fixing them before we get types sorted
out.

Absolutely not!

None of my concern. What happens from the point of sorting and onwards
is of no possible interest to me. I don't know or care where values
are "used" (parsed, combined, played, traversed, executed, whatever),
but I do know none of that should be expected to happen in the DBMS.

Sortability is not required. Atomicity is. The atoms only
need to be testable for equality. If you can do other things too, those
might be convenient and desirable, but not required.

XML should not be elevated over PDF. Nor over a string nor a convex
hull nor any other type. (It helps to consider geospatial types too.)

The DBMS should ensure that values really are in the domain, and to
validate other constraints involving them. I can't see that being
done in the near future--but it is a requirement.

Absolutely. That is the whole, entire, only point of defining database
constraints. It is to ensure that everyone updating it is working to
the same logical/conceptual model, for all time, everywhere. It is a
kind of contract. We don't do it very much yet and we probably never
will, but the DBMS needs to be able to do it.

Not my concern. All I care about is the problem stops being the DBMS.
Crap applications can come and go, I don't care, but I do care if the
DBMS gets the blame and we end up inventing horrible new
so-called database technologies that actually take a giant leap
backwards (or nowhere) because we didn't understand the problem.

Clearly you think I am going to get excited about this. I loathe and
despise XML. I think it is syntax gone mad. The reason I'd like to see
an XML type is because I want to treat XML "documents" as atoms (or
black boxes if you prefer, or maybe stool samples). I want to get the
data management job done without opening them up (or taking the lid
off).

And don't let's get too hung up on XML here. It's a good example to
work with but we could talk about much simpler "complex" values too.

Indeed, or a list or a bag or any other darned thing. I admit I don't
really understand very much about RVAs, but I also know there's nothing
in relational theory that excludes them. I think I'd find lists (or
trees) more useful more often, but your point is sound.

Incidentally, there is an OME example that implements lists as an Ingres
data type. I've been meaning to tweak Marty's Daitch-Mokotoff soundex
function to return a list.

I see exactly where you're going with that and I have the same problem,
but let's not go there just now.

You know, I think that's exactly what they'd do, and that's the bitch of
it, because I want them to sell it to people who *do* know RM/RT so that
they can compartmentalize XML and whatever madness comes next.

Ah, well I hope I've cleared that up.

Not my concern. All that matters is that there are people who do. At
one time it was because they didn't know better or were deluded about
what XML could do. Now the drivers are increasingly external. XML is a
nasty fact of life now.

Having a (suitable) box to hide it in will make my job easier.

I don't. We're on the same page brother.

--
Roy

UK Ingres User Association Conference 2011 will be on Tuesday June 7 2011.
Register at http://www.regonline.co.uk/ukiua2011