Hi all,

in my opinion this newsgroup should be a place to exchange news
about object database products and best practices for everyday use.

Some of the discussions in the past have also brought some technical
insight about internal implementations so maybe this newsgroup could
also be a place for vendors to develop new ideas together.

Unfortunately the level of discussions in this group has dropped
down recently. Some people are only posting here to discredit object
databases. I really don't understand their motivation.

Maybe these people simply have not come in contact with object database
usecases. Most of the recent discussions have been much too theoretical.
Companies need solutions today, immediately. They can not wait for the
"ideal" relational system, they need running systems right now.

Maybe we can write down some cases here, where object databases work
very good?

I would like to start off by sketching one of our customers, why he
decided to use an object database and how he is using it. I think he
has chosen the best alternative available today. If you don't agree
with me, please post a practical suggestion, where he is wrong and
what he should do different.


Case 1: Automotive
-------------------

Our customer is an IT department of a large automobile manufacturer.
In the past he has been relying on suppliers to deliver complete IT
solutions. With the increased complexity of the IT built into cars,
the decision has been taken, that IT competence has to be built up
in-house, to create long-term solutions at affordable prices.

Ressources are limited in cars and they will be for a very long time,
since reliable computer components for heavy duty conditions in cars
(heat, vibrations) will continue to be an important cost factor.
Components need to be small and light-weight. Important components
may need one or two backup systems, in case one of them fails.

The decision chain starts with hardware and the choice has large
implications on the rest of the system:


Decision 1:
"How much computing power will be available?"
We have signed an NDA, not to distribute any information but naturally
the systems will be a lot slower than todays desktop computers.


Decision 2:
"Which operating system will we use?"
Cars need real time operating systems. Here is a list of products for
further reading:
http://www.dedicated-systems.com/encyc/


Decision 3:
"Which programming language will we use?"
Java is a good choice for a couple of reasons:
portability, performance, vendor-independance, development environments,
third-party libraries, educated developers

Accordingly most automotive IT departments are planning to work with
Java. Some references can be found here:
http://www.osgi.org/


Decision 4:
"How do we store our data?"
As you can see, the decision for a database is located far down in the
hierarchy. This is a decisive difference to office applications.

I don't like the following statement but it does have some truth for
pragmatists that develop office applications:
"SQL has been around for a decade and it will be for another. Although
we can see the advantages of storing objects directly, let's not take
the risk of doing anything wrong. We start with our existing SQL system
and we will build the rest of the application around it."


The requirements in the automotive IT department are completely different:

(1) We start off with Java classes, how can me make them persistent?

(2) In the past C and C++ were used to store data directly. Can we save
ourselves some work, by using a database engine?

(3) Our system needs to start up very quickly. Anything above 1 second
is not acceptable.

(4) Speed, speed, speed is important. If the database engine consumes
too much time, the rest of our application will not work.

(5) Integration of components from other departments needs to be easy.

(6) We need to keep the memory footprint low.

(7) The size of the database file should be small.

(8) We can't supply a database administrator to travel along with every
car. The system needs to be robust and failsafe.


Here is our (db4o) solution to the above:

(1) One method call stores and updates any object:
ObjectContainer#set(object)
The structure of the Java classes is detected during runtime. Java classes
are used as the schema of the database. Fields can be added and removed at
any time and it's possible to continue to use the same database file without
a reorganisation run.

(2) Yes. The use of a database engine ensures that reading and writing
is transactional. It provides an interface to query for objects. It makes
it very easy to load and unload objects from memory:
ObjectContainer#activate(object, depth)
ObjectContainer#deactivate(object, depth)

(3) On the low-ressource hardware used we provided a startup time of 300ms.

(4) Our engine uses pointers with only one level of indirection for
transactions. It can be proved that this is the most direct access possible
so it naturally is the fastest.

Since database engine and application are both written in Java, the
complete system can be nicely profiled with a profiler like JProbe. We have
conducted some sessions together with this customer to search for bottlenecks
in eachothers code and we sure made him happy with the achieved results.

(5) Any Java library can simply be plugged into the system, even if it
was not written to be persistent. In contrast to JDO solutions, no
enhancers need to be run and no base class needs to be used.

(6) This leads me back to some of the points I posted yesterday:
- Our engine manages object identities and ensures that objects are only
loaded into memory once.
- Our reference system uses weak references that are automatically cleaned
up by the garbage collector when they are no longer needed.
- It is possible to unload transactional objects from RAM.
- We supply special database implementations of java.util.List and
java.util.Map that handle all database issues transparently in the background.
(used by this customer already, will be officially released with 3.0)

(7) Our engine needs hardly any file overhead for itself. I believe that
our one-level-of-indirection system is close to ideal. It could only be
improved by a compression system but this clearly is a no-op for
low-ressource systems.

(8) Our engine ensures ACID behaviour by using a 3-phase write algorithm
that can be safely interrupted at any point in time. If the system is
halted during commit, the commit process is resumed upon the next system
startup.


So far this customer is very very happy with our product and our service.
The only difficulty so far was persuade his business department that a
database even is a possible choice. "Database? Yes, that's what we run on
our huge servers in the basement. But we sure can't deploy a system like
that into our cars? Or can we?"

Yes he can!


Thank you for listening!

I would love to read some other object database usecases here.


Kind regards,
Carl
--
Carl Rosenberger
db4o - database for objects - http://www.db4o.com

"Carl Rosenberger" <carl (AT) db4o (DOT) com> wrote


Hi carl,

I'll chime in with my own experiences in the telecommuncations industry.
Several years ago I was part of a team resposible for the design, implementation
and delivery of an automated network recovery tool. The tools main requirements
were as follows:

- Detect network failures
- Determine impacted facilities
- Calculate failed facility failure points
- Calculate failed facility restoration paths
- Implement restoration paths on actual network elements
- Process the first 100 high priority failed facilities within
5 minutes after receiving the first alarm.

To give you an idea of the scale, the network consists of several thousand network
elements, with hundreds of thousands of interconnections between the elements,
which is a typical mesh network scenario. When a telecom network fails, money
stops flowing and customers get mad, so restoring the network to an active status
as quickly as possible is very important.

To meet the requirements, we needed to create and use a modified Dijkstra's
algorithm for calculating shortest path distance between the two nodes that
contain the actual failure point. Failures in the network were often created
when street construction crews would accidently dig up the fiber optic cable
and cut it in half. There's also another instance where a farmer digging a hole
to bury a dead cow also caused a network failure. But that's another story
for another time.

In this application, the main rerouting algorithm that relys on graph traversals to locate
a shortest path between any two nodes in the network.To implement the algorithm, we
needed to be able to quickly and easily discover all of the outbound facilities and their
endpoints, from any particular node.To discover a least distance path between any two
nodes, it means that you must exhaust all possible paths that exist in the network. The
algorithm is smart in that once a route has been located, all of the other routes that are
still under construction, that have a higher weight, can be terminated. Any route that has
not completed, but still has a lower weight than the completed solution, must be allowed
to continue running. (Multiple threaded solution).

In this highly specialized application, the RDBMS that we were trying to use simply could
not execute queries fast enough to keep the routing algorithm fed and still meet the time
requirement of getting the first 100 facilities restored in the alloted time. We ended up
switching to an object database (Versant) to resolve the issue. The graph traveral of the
objects in the database exactly matched the graph traversal that the algorithm needed to
do. The database performance was excellent. We easily met our performance requirement,
and in fact we exceeded the requirement.

In conclusion. This is a perfect example of application that was able to use an object
database to a distinct advantage. Could the application have been written using an RDBMS?
Yes, but at the time the performance of the relational database was just not good enough to
meet the requirements. Now, we could have used some tricks to make the performance
better, that is we could have cached the information from the relational DB in memory
somewhere, but that really starts to complicate things and it's just not worth the effort
when a better solution exists.

I have another example of where an ODBMS has been used very successfully, but i'll
save that one for later.

Cheers,
--Corey

Carl Rosenberger wrote:
In order to move our discussion to the more constructive direction, I
propose to define some problem that will involve both OO programming and
databases and then to discuss various ways of solving this problems by
using different types of DB and DBMS.
The OO programming language is not important for this discussion.

I propose the following problem which is common example in books on OO:
There are different geometrical objects (like points, lines, arcs, text
etc.) that can be drawn.
These objects are persistent and need to be stored in some intelligent
way for example in database.
In order to make this problem more interesting lets add some possibility
to modify such objects (by the terms of some gui that will be available
to user when he will press right mouse button).
In addition some logical and geometrical grouping must be allowed.
By logical grouping I mean something like - all objects that correspond
to some element in the picture. (This is hierarchical grouping.)
By geometrical grouping I mean something like - all objects in some
neighborhood (defined by some distance).

I think that it will be very constructive if we will discuss how this
problem can be represented in different DBs and how OO application that
will provide drawing and modification functionality to the user may
access those DBs.

PS. If somebody wants to propose something else he is invited to do this.

Best regards, Michael
http://www.yaooos.org

Michael,
An application I worked on had a component that some geometric features...
It was a geographic map of water pumps and pipes used to provide fresh water
to a medium size city. We overlaid the map with rectangles which
corresponded to worksheets which contained information about each pump,
pipe, reservoir, 'pressure area', etc. Users could also drill down to more
detailed maps.

Users had views to both access and configure these maps.

VisualAge Smalltalk was used for the client, GemStone/S for the server.
The map and rectangles were stored as objects. Maps were stored as SiteMap
instances with a ByteArray instance variable, which was a bit level
representations of a GIF file. The client software would render the map
image by reading the ByteArray as if it had read a real GIF file.

The rectangles were stored as instances of a 'HotPoint' object, which
contained upper left and lower right Point instances (much like a Rectangle
class). A Point instance is little more than an X and Y value, with some
convenience methods. It's trivial, for example, to create a rectangle from
these two points and check to see if a third point is contained within the
rectangle.

The HotPoint instance also contain callback methods which were triggered by
either right or left mouse clicks (for pop-up menus and drill down / open
view commands). The cursor would change shape depending on which type of
HotPoint the mouse was over (open vs. drill down).

Each SiteMap stored its collection of HotPoints. Each HotPoint knew its
SiteMap (we always used bi-directional links). When configuring a SiteMap,
uses would import a GIF file and add or select a HotPoint from a list. A
rectangle would be displayed over the map. They could then move or resize
the rectangle and set the actions of the HotPoint.

In effect each SiteMap was linked to its sub-maps by way of the HotPoints
that were used to drill down to more detail.
i.e. municipal site map -> region hot point ->
region site map -> pressure area hot point ->
pressure area site map -> pump hot point ->
pump worksheet (with pump data)

This was over five years ago, and it's still in production.
--
Bob Nemec
Northwater Objects

"Michael Groys" <michaelg (AT) alzt (DOT) tau.ac.il> wrote

<...>

Bob Nemec wrote:
I just wanted to proceed with our discussion of features and deficient
of different ways of data management on some concrete example. It must
be OO in nature (because of word object in the name of this newsgroup).
In addition it must be relatively simple although with some nontrivial
features. I think it may not be one of the projects implemented by us,
but rather be something neutral.
I don't ask people to write some code or create databases, just to
discuss how they would implement this problem in their way.
Then everybody can compare different approaches and see how good or bad
they are on the basis of concrete example.
This will be also very useful for other people that are starting new
project and need to decide how they will manage their data.

---
Best regards, Michael
http://www.yaoos.org

"Michael Groys" <michaelg (AT) alzt (DOT) tau.ac.il> wrote

One can neither demonstrate nor prove a generality by anecdote so I see no
utility in the endeavour.

If you want to move the discussion in a productive direction, I suggest you
learn the basics of data management and exercise whatever ability you have
to reason in the abstract.


VAR drawing REAL RELATION {
name drawing_name
, layer ordinal
, content RELATION{ id resource_id, item drawing_item }
}

Of course, that is only one proposal. I could propose an almost unlimited
number of designs based on the sloppy and incomplete specifications you
gave.


See above. I won't hold my breath.

"Bob Badour" <bbadour (AT) golden (DOT) net> wrote

Nobody is talking about the general case here. The subject is about "usercases",
that is, instances where object databases have proven better at providing a
solution to a particular problem than that of an RDBMS.

What we are talking about here are the realities of the current state
of DBMS solutions in the market today. Since you seem to agree with FP
that none of the current RDBMS offerings are true to the relational model,
perhaps you would agree that some problems are better solved with other
technologies, even though they may be flawed from a theoretical standpoint.

Bob Badour wrote:
Could you please be more precise:

Can a VAR have methods?
How would they look like?

How would you create an instance of a VAR "object"?

How would you draw one?


Kind regards,
Carl
--
Carl Rosenberger
db4o - database for objects - http://www.db4o.com

"Corey Brown" <corey (AT) spectrumsoftware (DOT) net> wrote


Such cases unfortunately exist but you need a good knowledge of the
fundamentals to adequately recognize them, and you completely lack
that knowledge.

OODBMS vendors try to cheat people to use their tools for problems
that are better solved with SQL DBMSs.

Carl's posts are a good example.

Regards
Alfredo

Alfredo Novoa wrote:
Instead of posting hollow phrases without any backup, why
don't you simply show us your solutions and explain why
they are better.

I have posted a couple of real-world problems and you have
not been able to provide one single practical example how
you would solve any one of them.

If you can only answer with insults, it makes me wonder if
you have any practical experience with software at all. If
your solution is better, simply SHOW IT.

I don't see where I "cheat" anyone. I only supply a solution,
how to persist objects. It's up to anyone to decide, if it is
useful or not.

Kind regards,
Carl
--
Carl Rosenberger
db4o - database for objects - http://www.db4o.com