dbTalk Databases Forums  

finding dupes and getting rid of dupes

comp.databases.filemaker comp.databases.filemaker


Discuss finding dupes and getting rid of dupes in the comp.databases.filemaker forum.



Reply
 
Thread Tools Display Modes
  #1  
Old   
peter
 
Posts: n/a

Default finding dupes and getting rid of dupes - 01-17-2007 , 12:06 AM






Hi,
just started working with FM8.5 this week, figured out how to find dupes,
however, here's where I need a little help. I've found all the dupes in a
database, let's call them "frames" and the list 200 items long. How do I
delete 199 of them and keep just one?
Thanks
Pete
--
P§³



Reply With Quote
  #2  
Old   
Remi-Noel Menegaux
 
Posts: n/a

Default Re: finding dupes and getting rid of dupes - 01-17-2007 , 02:16 AM






I'd say that you could either :
- script to find all those 'dupes', then 'mark' (ie put an 'X' in it) the
first one (in a new field called 'mark', after you'd erased first all
existing ones), then redo the find of 'dupes' and of 'mark' equals '=' (ie
nothing), then suppress all found record, then see all records.
- or do that by hand, creating the mark field, having nearby a script that
wipe out all existing 'X' in 'mark', reach all 'dupes', put an 'X' in the
first one, etc., see above.
I am sure it exists other ways...
Remi-Noel


"peter" <xvzex3qtgix (AT) verizon (DOT) net> a écrit dans le message de news:
K1jrh.14745$Ch1.5164 (AT) trndny04 (DOT) ..
Quote:
Hi,
just started working with FM8.5 this week, figured out how to find dupes,
however, here's where I need a little help. I've found all the dupes in a
database, let's call them "frames" and the list 200 items long. How do I
delete 199 of them and keep just one?
Thanks
Pete
--
P§³




Reply With Quote
  #3  
Old   
Matt Wills
 
Posts: n/a

Default Re: finding dupes and getting rid of dupes - 01-17-2007 , 08:43 AM



Are you saying you have 200 copies of the same record?

Find, Omit one record (the one to keep), then Delete All Records (which is in practice Deleting the Found Set, not all records in the file).

On the other hand, finding dupes using the ! results in every record that is a duplicate. there could be a hundred pairs of dupes, any number of "dupe groups" containing any number of records, etc.

Quick and dirty pseudo script:

Set error capture on

Loop

Perform Find [Restoring the dupes find]

Exit loop if Get ( FoundCount ) = 0

Delete a record. ( Any record )

End loop

Basically, your found set consists of nothing but records each of which is a dupe of some other record. Two, three or more in a dupe group, no matter.

When you delete a record that was found as a duplicate, then its twin is no longer a duplicate, and will not be found when the loop begins again; the found set keeps getting smaller and smaller with each pass of the loop. When there are no more dupes to find, the script ends.

The only issue is that you are absolutely satisfied they are duplicates, and you need only one of each, not caring which one you keep. Save a copy of the file beforehand.

Matt

On 01/17/2007 01:06:34 "peter" <xvzex3qtgix (AT) verizon (DOT) net> wrote:

Quote:
Hi, just started working with FM8.5 this week, figured out how to find
dupes, however, here's where I need a little help. I've found all the
dupes in a database, let's call them "frames" and the list 200 items long.
How do I delete 199 of them and keep just one? Thanks Pete

Reply With Quote
  #4  
Old   
eyebrown@mindspring.com
 
Posts: n/a

Default Re: finding dupes and getting rid of dupes - 01-19-2007 , 06:29 AM



I don't know if they still do this with current versions of FM (I'm stuck
with FM 6 do to size & complexity of my solution & lack of time to rewrite
three years of work). But the example given in describing scriptmaker in
the paper manual that comes with FM is just that. A basic dupe-killer.

This is a good choice on FM's part as it involves globals & looping & such
basic comments as flagging records as you go so you can retrieve your set
later. Essentially, find your dupes, sort by duped field, go to first
record, set a global ID to record ID, go to next record, compare global
to new ID, flag if different & reset global to new ID, etc.

You end up with all your dupes for each ID flagged (except one, which you
want to keep). Find the flagged records & delete all.

I've reused this basic script format for several dozen different jobs over
the years.

Also, opinions differ as to what constitutes a "dupe." Every single field
having identical contents? Just a key ID field being the same? Maybe a
combo, duplicate ID fields plus a couple others (AmountSold, for example)
but differing data in other fields do not count as dupes
(ModificationDate, for an obvious example)?

Finding these dupes is a matter of making a concatenation field of the
fields that you feel constitute a true duplication, and plug it into the
above.

I'll be happy to elaborate, if your scripting experience isn't up to
decoding my shorthand.

Steve Brown

In article <117943.CHKKFMGS (AT) news (DOT) verizon.net>, Matt Wills <Im (AT) witz (DOT) end> wrote:

Quote:
Are you saying you have 200 copies of the same record?

Find, Omit one record (the one to keep), then Delete All Records (which
is in practice Deleting the Found Set, not all records in the file).

On the other hand, finding dupes using the ! results in every record that
is a duplicate. there could be a hundred pairs of dupes, any number of
"dupe groups" containing any number of records, etc.
Quote:
Quick and dirty pseudo script:

Set error capture on

Loop

Perform Find [Restoring the dupes find]

Exit loop if Get ( FoundCount ) = 0

Delete a record. ( Any record )

End loop

Basically, your found set consists of nothing but records each of which
is a dupe of some other record. Two, three or more in a dupe group, no
matter.
Quote:
When you delete a record that was found as a duplicate, then its twin is
no longer a duplicate, and will not be found when the loop begins again;
the found set keeps getting smaller and smaller with each pass of the
loop. When there are no more dupes to find, the script ends.
Quote:
The only issue is that you are absolutely satisfied they are duplicates,
and you need only one of each, not caring which one you keep. Save a copy
of the file beforehand.
Quote:
Matt

On 01/17/2007 01:06:34 "peter" <xvzex3qtgix (AT) verizon (DOT) net> wrote:

Hi, just started working with FM8.5 this week, figured out how to find
dupes, however, here's where I need a little help. I've found all the
dupes in a database, let's call them "frames" and the list 200 items long.
How do I delete 199 of them and keep just one? Thanks Pete

Reply With Quote
Reply




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Powered by vBulletin Version 3.5.3
Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.