Hi, I have only been reading posts in some of the database newsgroups for a
few weeks, but I have been involved in database engine development for over
20 years.

In my world, where software is developed to solve problems, database engines
are purchased/used if they are a part of the solution. Then, the database
software's purpose is to follow the instructions of the application as it
accomplishes its purpose.

Perhaps I am misinterpreting many of the discussions I am reading, but it
seems that many participants define the world the other way around, where
the data model or the theory behind the model assume superiority over the
job to be done.

Is an application "wrong" if it uses a flat file as its database, when that
is entirely sufficient to do the job? Is it wrong if it uses a network
model database because space constraints rule out relational languages?

My experience shows that applications have needs for all kinds of data
storage and manipulation extensions. Its not the DBMS theorist's or
vendor's position to tell the application programmer how they must solve
their problem. That's not to say that development of theory and improvement
of implementation are not important. But the test of their importance is
when they are actually used to do something real. The best ideas and
implementations will prevail because they really help to do the job.

"Wayne Warren" <wwarren (AT) birdstep (DOT) com> wrote

I disagree completely. The database, like the applications, support the
business. It can, and should, prevent the application programmers from
making stupid mistakes, and from violating the information needs of the
business. Even in application code I firewall certain things, and make
classes immutable, to prevent stupid mistakes. The database also serves to
"formally document" the meaning of the data.

Yes. The business has a greater need: valid data. Leaving validity up to
each of the N applications that touch the database is foolish, as it simply
invites mistakes and misinterpretations and inconsistency.

Not necessarily, but there are costs that climb exponentially as the number
of applications rises above 1. Most businesses have important data that's
"touched" by more than one application; thus the need for consistency and
validation.

Which relational language? There are very low-footprint SQL engines. Very
few (meaning zero) relational ones, though.

What do you mean by "extensions" - to the application language?

Uh, if it means not telling them when they're violating the information
needs of the business, then you're wrong - it IS the business of the DBMS to
vomit back in the application developer's face, so to speak.

This isn't theory - DBMSs, and relational in particular, was created for
PRACTICAL reasons.

I consider telling the developer of an application that they're breaking
data integrity constraints that are important for year-end financial
reporting (one example only) "real". Don't you?

In other words, you're proposing a Darwinian approach to correctness? Most
businesses don't have the luxury of experimenting with every possible theory
and implementation.

"The job" is your weasel-phrase here. What job? Jobs in a business can
conflict. Why do you suppose the accounting, auditing, corporate security
and other departments exist in a company? How about labor relations?

Checks and balances. An app developer has to do a job, but if in the process
they violate data requirements of the company, many other jobs could be in
jeopardy.

- erk

Correction below - I misinterpreted you.


"Eric Kaun" <ekaun (AT) yahoo (DOT) com> wrote

What I should have said:

It is the data theorist's job to suggest good ways to model data. It is the
DBMS designer's job to model the structure and behavior of the DBMS that
will process data according to some theory (like design, the theory is
there - you can either be explicit or not, or use a solid theory or not, but
the theory is always there).

I don't recall any theorists anywhere wrestling developers to the ground to
prevent them from being stupid. But it is certainly their prerogative and
within their purview to mention the stupidity, and to present a better way.

In response to the implicit "there is no one right way" and "there's more
than one way to skin a cat" argument: relational has presented arguments as
to why it's better, and illustrations of relational algebras and calculi
indicate their expressive power, their useful properties, and the value of
the principles upon which they rest. I have yet to see another theory do the
same. Compare the writings of the relational theorists to those attempting
to formalize nonsense like XQuery, and describe to me what additional power
the theory has. It's excess complexity with no increase in power; at best,
XPath expressions have a place as operations on a data type (XML type, that
is).

- erk

"Eric Kaun" <ekaun (AT) yahoo (DOT) com> wrote

<...>

Sure... I don't think anyone on this forum has argued that relational does
not have a solid theoretical base. What has been said is that a real world
implementation may benefit from using other tools for persistence (like an
OODB). These tools may not have the full expressiveness of a relational
solution but the application may not need it ... and the OODB probably
solves a niche problem better than a general relational solution.

If I had many applications hitting a common data store, I'd probably also
want to enforce data integrity rules in the DB. But if that's not the
situation, why introduce the overhead of having two areas semantic control
(constraints in the database and behaviour in the application)?

In our application, we program with objects (not 'data' and 'programs') so
*all* of our behaviour is defined in our object database. 'Applications'
are just different views of the same object model. I'd rather do that then
split behaviour and data.
--
Bob Nemec
Northwater Objects

"Eric Kaun" <ekaun (AT) yahoo (DOT) com> wrote


developers to the ground. Your method is verbal, and it happens all the
time. Subdued developers, who could otherwise complete their assignments,
have become convinced that they are not qualified to use a DBMS.

superior to offer. But I qualified my comments as referring to "my world,"
which is a bit different than yours. My world is not the Enterprise with an
array of business applicaitons and ever-changing requirements, or rogue
developers.

Database engines are being used in places where they weren't practical
before, in embedded computers or within industrial processes. Their
services are useful in real-time systems which until recently couldn't use a
DBMS because of storage or performance limitations. They are also being
used in mass distribution, where its not practical to license the
full-featured but expensive DBMS products.

I will welcome the day when relational solutions are available in these
other places. But practically speaking, the simpler DBMS solutions are the
ones that fit, and happen to work (do the job?). Technology will eventually
catch up to the theory. In the mean time, the small footprint computer
systems of today are much like the big computers of the '60s, and some of
the same old (flat file, hierarchical, network) database solutions are the
best answer.

Eric Kaun wrote:

Eric, if you're talking about things like integrity constraints,
you have my agreement. Much like a compiler prevents you
from writing invalid constructs in your code, the DBMS
should prevent you from violating referential integrity.

On the other hand, I have agree with Wayne that, as
programmers we have to solve a certain set of problems
which are defined by the application at hand. If, at
the same time, we can address problems of data redundancy
et. al. which relational algebra addresses, well and good.

As to stopping programmers from making stupid mistakes,
you might as well try to stop world hunger, baldness
and "the heartbreak of psoriasis" at the same time.
As one person put it ...

"The two most ubiquitous things in the universe are
hydrogen and stupidity, not necessarily in that order."


This is the referential integrity argument again. These checks
could be left to the DBMS engine or encapsulated in library
calls which everyone *must* use on a discplined project.
Whether or not this library code would be more or less
efficient than the DBMS-engine is debatable.

This is a very real problem which, to my knowledge has
never been solved in the real world, although many
attempts have been made. If you are developing a
"one of" application, for a fixed price and a deadline,
the use of flat files may be exactly the right choice.
Once you get past 1 application needing the same data,
you are right (but exponentially is probably stretching
the point).

Note that sharing of data between applications need
NOT be instantaneous. A nightly download is quite
acceptable for many cases. Each case has to be
examined individually. Primary-secondary relationships
have to be well-defined.

At the risk of putting words into Wayne's mouth
(sorry if I misinterpreted you Wayne), using
the currently available SQL implementations
and having the need to apply multiple indices to
tables could use up considerable space
over and above the basic needs of data
storage. In a CODASYL model, where access
paths are well defined and known a-prior, the
only overhead is for the pointers to the related
records. (G_d help you if your navigation ever
needs to change though, but let's not bring
up the network vs. relational debates of
20+ years ago again.) This space limitation
may well constrain the choice of DBMS engine
to something other than relational.

Then again, 1NF set of tables is also
wasteful of space, so you have to make
tradeoffs.

Here, I will take a middle ground. Something like
the relational model provides a starting point
which can be used for analysis. Using that
base of knowledge rather than reinventing the
wheel regarding data-modeling is a "good thing".

Assuming that you then must implement the physcial
data structures on disk (e.g.) exactly as the
model dictates is often overkill. My philosophy
is probably similar to Wayne's. Do what's necessary
for the current job at hand.

So, who write the integrity constraints? Another programmer?
They are just as likely to be wrong (initially) as the
application developer. (Granted, usually these are
checked much more throughly than application code.
But that's a process issue when you come to think
of it.)

And each of these departments has different requirements
on how they use the intersecting set of data, and at what
frequency. Try telling your customer for your application
that the reason it's going to take 10+ minutes (e.g.)
to update an Employee record is that someone from
accounting is running the year-end financials.


--
"It is impossible to make anything foolproof
because fools are so ingenious"
- A. Bloch

And I didn't see this followup before I posted my
rim shots, either


Eric Kaun wrote:

Maybe not in this forum there haven't been, at least
recently. But I have worked in places where the
so-called Data Modeller had the final say as to
the layout of the database. In some cases, it was
the data modellers who were stupid and would not
budge from the relational algebra even when
presented with de-facto evidence that their model
could not meet, for example, performance
requirements. I guess that's one of the reasons
I still have a job. Fixing the physical data
layouts to match actual access patterns and optimize
those.


--
"It is impossible to make anything foolproof
because fools are so ingenious"
- A. Bloch

You assume that the application owns the data, but this is a realy
rare case. For common data this would trigger a import operation of
the data when the application starts and an export operation when the
application ends as soon as a second application uses the same input
data or needs the result of this application.

They start from the point that the data represents the business and
that there are a lot of different applications which work on the SAME
data pool. In this scenario you tune the overal acceess speed and
not the access speed of a single application.

As long as the application owns the data, i.e is the only one that
uses it, it can store the data in any form it wants.

Regards from Germany

Franz-Leo

Franz-Leo Chomse wrote:

Quite reasonable, actually.

I think this points out wide differences in experiences
between the different folks here.

From one standpoint, the concept that the data
represents the business is very much valid.
What many folks do, however, is make the
(unsubstantiated) leap of faith that every
application which uses any of the data must
have the latest "up to the second" data
to work with. (I am not implying that you do,
Franz-Leo, just that many people do.)

For certain classes of applications, it
is perfectly acceptable to use "last
night's data" (or pick another time
period, if suits you). Thus, what you
suggested above regarding export/import
is quite a viable alternative for these.
(Caveat: the primary-secondary relationship
between data source and data sink should
be well understood and never violated,
but that's another issue.)

For certain other classes of applications,
having to access the huge corporate data-store
is just not practical. As an example,
assume an application which is generating
revenue for the business, such as a
telephone switching system. When a call
is placed, some sub-set of the callers and
called party's data needs to be available to
this system in order to complete that call.
Every call completed generates revenue for
Deutsche Telekomm (my apologies if I
misspelled that, Franz-Leo) therefor
the company wants & needs to have these calls
complete as quickly as possible. Many,
if not all, of these kinds of applications
have a need for reading (order of magnitude)
these applications does *NOT* use the corporate data
repository to do its job but has a copy
of only the subset it needs (and may
actually have more information than
the corporate data repository, e.g.
speed-dial lists, etc.) In this case,
I think you would agree, that the
application needs to behave /as if/
it owned the data it was using in
order to meet its requirements, and
thus is free to choose any data representation
or storage scheme it deems best for
its needs. (It, in fact, may be
implemented as several embedded
processors within the physical enclosure,
each of which operates on a yet a smaller
subset of the switches data and each
of which may have its own representation
of the data.)

(Note: This application also
generates billing records which are
forwarded, at periodic intervals,
to the corporate repository so that
the appropriate accounts-receivable
application may issue bills and so forth,
so that it indirectly updates the
corporate data.)

This is quite a different environment
from what I think of as the corporate
data repository. (Correct me please
if I make any mis-statements below.)
There, applications such as accounts
payable, accounts receivable, sales,
inventory control, payroll, etc., etc.
are all using intersecting sets of
the same data. In fact, in order
to ensure consistency of the "books",
it is almost a given that they *have*
to be accessing consistent data, thus
must be accessing the same (logical)
data store. In this case, as you say,
the data store is not optimized for
any of the individual applications,
but for overall access speed.
(Sort of like politics, you make
everyone a little unhappy, but no
major group extremely happy or
unhappy.)

Sorry for such a long post, everyone.


--
"It is impossible to make anything foolproof
because fools are so ingenious"
- A. Bloch

"Bob Nemec" <bobn (AT) rogers (DOT) com> wrote

Persistence tools are one thing, but when the "tool" impacts the data model
itself, there's a problem. There are OO-SQL mappings which rely on a SQL
data store, and thus preserve a model which is lame but better than OODBMSs.

I would argue that most programming languages could benefit greatly from a
built-in relational algebra, not just for data processing but for (drumroll
please) logic itself. But even if App X doesn't need relational, what about
Apps Y and Z and... ?

Solves one problem, but creates others.

I can't think of a single case where I'd want to make that gamble,
considering that the cost of change would be very high. Every tiny little
app that I've ever written has grown - although in actuality, it was the use
of the data that grew, as additional apps wanted to tie in and use what App
X started. By the way, that's a sign of success ("they like your data! they
really like your data!"), and I certainly prefer to design for success.

Behavior is different from control, and you put constraints in the DB so
that multiple apps don't step on the data and each other.

Objects are data and programs - they're just welded together, sort of like
in the movie "Tetsuo: Iron Man".

So what does a view of an object model entail? A root object somewhere?

Behavior and data are split, naturally. Objects function well to the extent
that they're considered values of types, stored in variables. But object
trees are hard to manage, and when you couple that with the fact that not
all operations are just naturally part of a one object, but are in fact
multi-class polymorphic, you get a mess. Sure you can introduce helpers and
factories and everything else to ease this, but why bother?

- erk