I had thought that all software engineers were familiar with what analysis
is, and how it differs from design. But perhaps not. There have been a
couple of basic questions in this newsgroup, from people that seem like
serious professionals, that suggest that analysis itself is widely
misunderstood.


Given the number of large projects that have begun with inadequate analysis,
and have ended in disaster over the last decade, perhaps there's a
widespread ignorance of the fundamentals of analysis.

I'm hesitant to offer a definition off the top of my head, because it will
surely be torn apart by the usual gang of vultures. In the meantime, I'd
like to hear from everybody with a degree in software engineering. Did you
ever take a course on analysis? Or, alternatively, did you ever take a
course on methodologies that put a strong emphasis on analysis?

Have any of you ever undertaken a large scale database design project
without doing any formal analysis, or just by writing down the requirements
in a doc? What happened after that? I'm not talking about a little
database with 20 or 30 columns. I'm talking a database with upwards of 300
columns and a good number of tables.

On Dec 2, 4:23 pm, "David Cressey" <cresse... (AT) verizon (DOT) net> wrote:
What, here in c.d.t.? Noooo, that can't be right. :-)


I have a bachelors in CS from UC Berkeley. Never took a class
in analysis. Never really heard of it until the last 5 years or so.


Never took any methodology course.


Well, at my last job, most of what I did over the years
involved working with this one big database. It had
roughly 75 tables when I started, and many hundreds
when I left. Data size in terabytes; revenue in billions.

Since I don't know what analysis is, I don't know
whether I did any or not. :-) However it was pretty
rare that I ever actually did anything so heavyweight
as to write down requirements in a doc. Usually I
just thought about what was needed and wrote code.

There were two other large scale projects I worked on,
in a role that I guess could be called "chief architect"
or some such thing. In both cases I did all the schema
design myself, not wanting to leave this part to junior
engineers. I did my designs by writing DDL in vi.


Uh, worked fine?

The big problems tended to be rapidly changing requirements.
(Not that we misunderstood the requirements in the first
place, but rather they changed after the initial design.)
Oh, and Sarbanes-Oxley.


Marshall

Quoth David Cressey:
I'll have a go, then: Analysis is taking something apart (to see how it
works). Whereas design is putting something together (to make it work).

No. Nothing that covered analysis /thoroughly/, at least, and certainly
not /formally/. I've learned a few diagramming notations in my time, but
I've never had analysis presented as a science as opposed to an art or
craft.

I have a database of currently 102 relvars with in total 590 attributes.
No formal analysis, whatever that means. I drew some pictures in the
beginning for communication, but once I had a prototype, it was simpler
to just show, tell and ask. People understand tables just fine. What do
you mean, "What happened after that?"
--
Jon

"Jon Heggland" <jon.heggland (AT) idi (DOT) ntnu.no> wrote

I like this a lot better than my own attempts. I this case, I think it
means "taking the problem apart". Breaking the problem down into pieces
that are easily assimilated. The term "easily assimilated" has more to do
with psychology than logic, so we should be prepared for the general
complaint that "analysis is subjective".

In the above terms, the opposite of analysis is synthesis. I like to think
of the overall life cycle as consisting of "problem analysis and system
synthesis".




Maybe it is an art or craft rather than a science. Maybe presenting as if
it were a science is missing the point.
I think you answered the question. You raise a very important point:
prototyping
and successive apporximation. You didn't say successive approximation, but
I'm inferring that from the word "ask". Prototyping has always been an
attractive alternative to analysis.

I would certainly prefer prototyping to what has been called "analysis
paralysis". In fact, I'll go so far as to say that prototyping, done
properly, is itself a form of analysis. It blends analysis with design in a
ways that classical analysis does not. However, I've seen prototyping done
wrong, and that can lead to disaster.

How do you show people a table? Do you se a diagram? Do you show the table
through the lens of the application you are building? Do you give them what
MS Access calls a "datasheet view"?

Do your people understand foreign keys just fine? My experience, going
back to the 1990s was that people who had not worked with databases did NOT
understand foreign keys just fine. That includes people with 20 years COBOL
programming experience and who were in other respects highly proficient.
Managers also couldn't get "the big picture" from a datasheet view.

My biggest success with diagrams did not come from the analysis of a
proposed system. It came from reverse engineering an existing database back
into ER form, and using that diagram to communicate with managers and
programmers.

I used a tool to perform the busy work. It took me all of one day to
extract the metadata from the devlopment database, generate the ER diagram,
make subsets of it for presentation purposes, and copy the result to
transparencies (yeah, I know, that's primitive). I will admit that MOST of
the communication was in the form of questions and answers in plain English,
rather than the diagram itself.

Where the diagram added value is that it kept the conversation moving
forward, instead of going around in circles.

The database was about 100 tables, and maybe 600 columns. I forget how many
entities and relationships I ended up with, but it was about 100 in total.
That's because every table describes an entity or a relationship. There are
a few extra relationships hidden in entity tables, and there are few tables
that are "about nothing" in the Seinfeld sense.

Before I did this, the only person who understood the database was the DBA.
After I did this, almost all the stakeholders understood it. This was not
analysis. The database was about customer relationhip management in the
cell phone industry.


When I built new databases, much smaller and simpler, I used diagrams for
my own purposes. It was much easier that keeping the details in my head,
or writing the details in formal English in a doc, or embedding the details
in code. I also found that diagrams were much better and faster at
communicating with management than "progress reports".

David Cressey wrote:

Since it was the telephone/telecommunications industry, is it fair to
say most of the managers had an engineering background?


I suspect you have worked with a different sort of management than I have.

Marshall,

I only have one book on analysis on my bookshelf. It's "Object Oriented
Analysis" (2nd edition) by Peter Coad and Edward Yourden. My copy goes
back to the early 90s. The state of the art has almost surely progressed
beyond this. I would recommend you get and read this book. An alternative
would be to get a successor book, one the authors might recommend, if they
are still around. Again, the state of the art has almost surely
progressed.

From your posts some 3 year back, I know that you are something of an
"object oriented programmer in recovery". And, even though you are able to
pass in c.d.t. as just another relational bigot, I know you are little more
open, in secret, to object oriented thinking than some of the rest of us,
including me. I'm not very open to object oriented thinking, but that
didn't prevent me from appreciating OOA.

What's interesting about OOA is that it does NOT suffer from the
"object-relational impedance mismatch" that has plagued people who have
tried to build systems using OOD and RM together. I found this book very
easy to accept, without abandoning my preference for "relational think"
over "object think". It helps that the authors understood and appreciated
the way "the database people" go about understanding the subject matter.

Here by the way, is the definition of "system analysis" they offer for
starters, from DeMarco[1978]: "Analysis is the study of a problem, prior
to taking some action."

This isn't perfect, but I'll offer it in preference to my efforts.

They contrast OOA with functional decomposition, the data flow approach,
and information modeling, even though they borrow freely from all of them.
They deal extensively with the problem of changing requirements.

You seem to have gotten a long way with no formal training in this area.
You might get even farther with some reading. Warning: the input to
analysis is almost entirely psychological. The output, however, is expected
to be logical, or at least useable by logical people. Second warning:
analysis is not design. I keep repeating this, but we keep forgetting it.

"Bob Badour" <bbadour (AT) pei (DOT) sympatico.ca> wrote

No, it is not. There were plenty of technical people in the IT department,
but they were almost all working in the "switching group". This was a group
that tracked the behavior of the cell phone towers, and captured data about
usage, as well as data about malufunctions. The management in that group
was almost entirely engineers.

The management in the group that handled the customer database was business
oriented, and had almost no engineering background. Those that were former
IT technicians had grown up in the files and records era, and both the
relational model and SQL were largely uncharted territory for them. They
relied on the programmers who worked for them to have current technical
knowledge.

The programmers had a rudimentary understanding of SQL, but they didn't
really understand databases.

Probably true. The interesting thing to know is which way the trend is
going. Is management becoming more technically aware or less so? Given how
fast things are changing these days, are managers remaining technically
current as well as honing their management skills? Do they have any
management skills worth honing?

Do they understand the fundamentals of data modeling?

Quoth David Cressey:
Yes, I was thinking about analysis and synthesis when I spoke my piece.

I was half hoping that you might know of any science in the area, since
you used the term "formal analysis" earlier, and seemed surprised that
there is confusion about what analysis is. Oh well.

"Approximation" sounds strange to my ears in this context; I usually
talk about "iterative development". Which goes almost without saying; I
can hardly imagine building an information system any other way. The
users have hardly any idea what they want until I show them something.

I'm not familiar with Access. My tool of choice, Dataphor, creates GUIs
from table (or rather view) definitions. There isn't any clear
separation between the database and the application(s). So I show them
the actual application, which shows actual database tables/views quite
plainly. Sometimes I draw sample tables (with example data, not just
headings) on a whiteboard or something. And sometimes I use an ER-like
notation (Carlis and Maguire's LDS), but with a strict and simple
correspondence to my relvars. The relvars come first; the diagrams are
simplifying representations of them.

I don't consider foreign keys all that special. It always annoys me when
someone says that foreign keys signify relationships. They don't; they
/constrain/ relationships. Relationships are given by attributes being
defined over the same domain (although this makes far more sense if your
DBMS has proper domain support). Anyway, I don't feel the need to talk
much about foreign keys. I talk about identifiers (key constraints), and
say that this column/attribute is the same as that.

That fits with my experience, too. But "back into ER form"? Did it start
out that way?

Ditto. Except, y'know, Norwegian.

Yes. One must have some sort of artifact to point at and scribble on.

No? I'd say it was, but an analysis of the database, not the business
problem.
--
Jon

"Jon Heggland" <jon.heggland (AT) ntnu (DOT) no> wrote

I don't think of "formal" as a code word for "scientific". In fact, I
don't even think "engineering" is code for "scientific". Science is about
discovery. Engineering is about invention. There is a lot of overlap in the
language, the tools, the method and the mode of thought. But they aren't
interchageable.

I'll accept "iterative development" as better sounding than "successive
approximation".

And I'll accept that information systems evolve along with changing
requirements. But
I'll also claim that some good analysis can accomplish in weeks when take
months worth of iterative development. I've seen analysts that are better
than I am doing their work. It's really impressive, except that the
interviewees are generally nuimpressed. One manager's summary: "He's not
very bright, but at least he's thorough." What the manager didn't know is
that the analyst deliberately asked the same question over and over again in
different words. If he had gotten different answers each time, that would
have told him something he needed to know.

Fair enough. That's really what I was after. I hold no brief for Access.
It's just that it existed on almost every desktop that I used at a client's
site, it was easy to link to "the real database", and it had a very easy
learning curve, from my point of view.



It would be interesting to know if there is a way of deriving relvars from
some form of data analysis that doesn't involve ER. I am convinced that a
set of relvars embeds design decisions, but I don't know from direct
experience. What do you say?

The word "signify" strikes me as odd and stilted in this context. I prefer
to say that "foreign keys represent relationships."


No, I didn't mean "back" that way. What I really meant was "backwards".


Well, yes. After a while, the database becomes "a part of the system". At
that point, system analysis includes database analysis. Since I came to
this case late in the game, the database was already "part of the
environment" a far as most of its stakeholders were concerned.

David Cressey wrote:

Pure science is about discovery. Engineering is about applying science
whether to invention, re-invention, patent subversion, measurement,
design etc.

[snip]