On Jan 10, 5:37*pm, Ross Ferris <ro... (AT) stamina (DOT) com.au> wrote:
In many cases I have no choice but to use a complex CALLX due to
the disparity of data or a lack of pre-summarized data. In most cases
I can get by with translations and/or a simple CALLX to perform some
simple logic. All of the CALLX ordeals obviously slow down SSELECTs
and affect performance. For some instances I just toss another data
colletion script in the file save app. For others I insert modules
into the process points so detail can be captured live and recorded
while being summarized in other (new) places at the same time.

The reason I'm thinking AJAX is because I can paint a cohesive
picture on the page with IFRAMEs, glue some bits together and add
glitter with JQuery toys, while keeping logical code bits completely
separate in the server. The back-end side can be meshed together and
tied to our core system with state data. I use AJAX review/update
modules already and have multi-media item locking integrated into a
few of our green-screen apps. Some pretty complex operations can be
done without making the user skip from page to page, but the drawback
could be locking everyone out of a huge list of part numbers if a user
doesn't click the [Exit] button on the edit page. It's not really an
object-oriented solution but I think I could get what I need done
without having to write a monolithic GUI app in .NET. A benefit over a
GUI app is that If it's designed well it would be easy to add new data
and report modules to the framework. No real UI work would need to be
done.

Not having the time to design and build all of that is the reason
I'm looking for something off of the shelf. If I have no choice, I'll
build something to fit our needs..when I get time. I can paint a more
descriptive picture once I determine which business processes need
better update and reporting tools and the scope of the work that needs
to happen. I have a grasp of the immediate needs, but when you start
showing off tools like that everyone else wants a piece of the action
in their own way.

GlenB

I'm going off topic here but...

The situation you describe seems to me to be the biggest flaw in the
MV model. It just doesn't seem to be designed for large databases
with tens of millions of records and thousands of tightly related
fields across files. I posted a scenario a while back asking for help
with the design of an app that has similar requirements and there was
absolutely no feedback. I'm guessing that means no one else could
propose an effective solution to the problem either.

In short, massive volumes of data need to be reported in different
ways. To accommodate the transactional and reporting scenarios we
break the data into different files, but that becomes a mess with data
now too fragmented and we never really work out all scenarios ahead of
time. Indexing helps but it doesn't fix an incomplete design.
Ultimately, after development we have new requirements for which the
model wasn't designed and some redesign seems to be required - but
that task is overwhelming.

One tool that can help with this problem, but it doesn't solve it, is
file Parts which is supported by a few MV platforms.

The traditional approach to relating data across a lot of files is to
use cascading Translates. That runs into problems when multivalues
are involved, and (while I guess this is just the nature of what we
do) all of the relationships amongst data need to be described in
excrutiating detail, which get painful when working with dict items.

Rather than long chains of Translates in Correlatives, the only
mechanism that seems to help across platforms is to use dict items
that execute BASIC which understands the file structures and can drill
across files. Code can be re-used and it's centralized so that one
change affects a lot of files. That's more of a relational approach
with Stored Procedures. Coding and maintaining that kind of
environment is very tough, and (probably depends on platform)
performance with that structure can be very poor.

T

On Jan 9, 10:09*am, "Jeff Caspari" <mu... (AT) idt (DOT) net> wrote:
All I can say is, "we'll see." If it was FOSS I'd start a project and
let people jump on-board. Right now, this is an in-house commercial
coding project so I have to handle it differently. I'm not a contract
coder and my FOSS development time has pretty much disappeared. If you
think this is something that a bunch of developers would be interested
in fostering and supporting then I'll help get it rolling. I'm sure
there are a few product vendors that would be eyeballing it closely to
see what comes out of it. Frankly, though, if I find something already
available that I can work with I'll be devoting my time there as much
as I would like to see an open source solution be born and mature.

GlenB

On Jan 12, 1:29*pm, Tony Gravagno <tony_grava... (AT) nospam (DOT) invalid>
wrote:
There is no flaw in the data storage layer, but the data layout
itself could be counter-productive depending on how the data is going
to be used. The same data relationships must be dealt with in a flat
or object-oriented environment regardless if it's normalized or nested
on the storage side. The development tools normally dictate how the
data is pulled and used, right? SQL tables can be complex objects in
PHP, but MV files can be too.

Yes, to the fact that transactional recording lends itself to
granular data files. The opposite is also true in many cases and then
you end up with no detail data for historical review and analysis.
Which is worse?

That would be nice on D3 and I am not sure which platforms support it
or the limitations. Maybe D3 already does and I'm just clueless?

Ugh.. I try my best to avoid cascading translates and I don't really
like them on normal files but it's a quick and dirty option. I'll
build a secondary file and update it when possible.

It's tough to employ modular code like that since each CALLX
requirement is very unique to a specific business need. Sometimes you
can reuse core logic code but I've never really found myself calling
subroutines from within a CALLX routine. A few specific examples of
where I had to use sub calls within a CALLX has to do with calculating
projected stocking levels based on a fairly complex engine. The same
engine is called for different warehouses and thus each one is a
different dict item. If you were to, in my opinion, abuse CALLX then I
can definately see cases where the same code would be used over and
over again in a bunch of dictionaries from lots of files. If the same
code is used over and over again in CALLXs eveywhere then, again in my
opinion, there is an inherent lack of data support.

GlenB

D3 doesn't support "part files" --> they were originally created to
overcome 2Gb OS file size limits in environments like U2 & jBase ....
after a while, it turned out that they could be put to other uses

The closest approximation I could gice nwould be a single file with
multiple data portions ... with part files in other systems, the
mechanism to decide which "part" a subfile belongs to becomes part of
the file --> your view of the world is a single file, but with
knowledge of the partitioning logic, you can be :smart" and process
queries across discrete part-files.

You can simulat this (at the cost of storage .... but hey, that is
cheap these days) by having, say, a CALLX that updates a single "mean
mother of a consolidated file" from related part files --> knowing the
partitioning method (lets say, date, for a transactional history file)
you can get bthe same "benefit" by running discrete queries against
(hopefully) appropriately segregated part-files .... and in a worst
case scenario, you select/list/sort that big mean mother, probably
with a few indices for known/common "exceptions"

Actually, "good indexes" can help you achieve most of the same
benefits typically associated with a PART file from a performance
perspective .... but when that sucker needs a resize or backup/
restore, you need plenty of time on your hands!