Let's talk about an entirely different approach to deleting duplicate records which actually does delete records. What we're going to do is Export the non-duplicates to a separate file, delete all the records in our current FileMaker file and then import the non-duplicates back into our FileMaker file and be left with non-duplicates. Since we're deleting all records it's very important that you do a backup here. In fact, when I was recording these tutorials I made a few mistakes myself and was glad I had the actual backup. It'll really help you out in this situation. It's a much bigger scenario than if we accidentally delete one or two records. We're talking about possibly deleting all records, so a backup is really essential here. And we also have to consider what about related data? What if we have an invoice that's related to one of the duplicates that gets deleted, what happens to that invoice? Well, we've set up an example so you can see that. So let's go ahead and talk about what feature we're going to use to export the non-duplicates. Well, there's a feature in FileMaker that allows you to export Subsummary data to a text file. You can export to a PDF already and that's a much better report, but what if you want to use that data somewhere else? PDF doesn't work real well. What if you want to take that data into Excel and manipulate it further? Well, this feature works great. You can export to a Subsummary Report and remember, when you're exporting subsummarized data what happens is it groups your data. Let's say we sort it by Dup Check. It would sort all of the Billy Adlingtons into one group and export just one record, so you want to remember how that works. We're going to get less records when we export using this technique. So let's demo it manually so you get the idea here. We're going to go into Records and choose Sort Records. We'll Sort by dup check, go to File and choose Export Records. We'll call this dup.fp7 and make sure we're going to FileMaker Pro format although that's not the only format but it will make it much easier. We'll Save and you may have not noticed this before but if you sort before you come into Export all your sort fields will show here. If you don't check any it'll export as normal, in other words, it will export all records. But as soon as you check this it's subsummarizing and you'll get less records. So we're going to clear all this out and then go to our table and move all of our fields in here. Now you really don't need all of the Global fields and the Calculation fields, they're pointless, but we're going to go ahead and leave them in here just to do it more quickly. You can take the time to remove these from your export and possibly make this technique work faster because you really can't import Calculation fields anyhow and Global fields, you know, there's no real data in them anyhow. So we'll Export. You can see the file automatically opened up. We have 48 records of our original 52, so it deleted those duplicates. So let's go write our script. We'll create a new script, call it Delete Duplicates Export, and we can put in Allow User Abort and all that stuff. You already know how to do that. Let's get down to the meat of the solution which is we're going to start off with Perform Find. There we go, move that in there. And the reason we're going to use this is we want to make sure that we don't export all of our records. If we have a million records it's going to take a long time to export and a long time to import, especially on import when it has to go ahead and index all the fields it's importing and we suggested this Perform Find with our Looping method because we don't want to loop through a million records. Well, it's even more central here because the Import and Export is going to take a much longer time. So we'll Specify our Find Request and because I did the manual Find first well, it's going to have it in there already, but if you didn't do the manual Find all you have to do is select Dup Check and put an exclamation point and hit Add and then you're done, that's it. Now it's going to Find and we're assuming here, we're not doing any error check and we assume we're going to find duplicates but you probably want Error Checking in here or there's no point in going over it again. We've already gone over it a million times already. So our next step is going to be Sort Records. We'll move that over, Perform without dialog, Specify sort order, it's already in there, dup check, need to make sure we do that before we export records. Export records, not Export Field Contents. Perform without dialogs, Specify output file, make sure it's FileMaker Pro. We'll add the file that we export. Here's where the benefits of having manually done it, we can simply click on that. It puts it in there and we're ready to go and we don't have to worry about the path, it figures the path out for us. And it could be more complicated than this. This path represents our Desktop which is very easy to type in. So we'll click OK, Specify the export order, it's already in there because we manually did it, so doing these things manually can really save you some time. Then we're going to Delete All Records and here's where I want to warn you again, make sure you make a backup. You can't undo a Delete All Records, or even a Delete Record, but a Delete All Records is much more crucial to make sure you have a backup on that. And then we can finally import the records back in. We'll move that in there, Perform without dialog, Specify data source, choose FileMaker Pro and since we know the path we'll just type file colon dup.fp7, click OK. Specify the import order and, you notice the Source is not up here and why is that? We specified it. We probably typed it in correctly. Let's check that out. Let's go back here and let's Add File and you'll see that dup.fp7 is gone and remember, when we did that export right up here, it said: Do you want to replace? But it actually deleted it because it didn't have any file to export. Why it does that I don't know, but it did delete it. So to make this much quicker we're going to go ahead and save this, close this, quickly export manually again. We'll go to dup.fp7. Don't need to automatically open, Export, go back into our Scripts and this is going to save us a ton a time, look at this. We go back into Specify now everything's there, all I do is choose matching names, make sure it's on Add New Records, we don't want to perform Auto Enter because we don't want to change our modification information or our creation information and that's it. That's the whole script. We'll save that, close that and now what we're going to do is run that script and see what happens, so make sure you have a backup here. And you see we have 3 records that were actually de-duped. If we Show All Records we'll see we now have 48 total and here's the problem that I was talking about. Let's go over to our Invoices and let's look at the last invoice here. Now, I had Billy Adlington but the second Billy Adlington on here. You notice once I deleted this duplicate record this information is now gone here. He still has his products he bought but really what we wanted to do is merge the old Billy Adlington with the new one that, we kept the old one, we wanted to merge this and so that what we really wanted to have happen was for this, for it to substitute the Billy Adlington we kept in for this invoice so we didn't have that problem with our data. So you really have to consider what about the related data when you're using this technique.
TERMS & CONDITIONS OF USE
BY SUBSCRIBING TO THIS SERVICE, YOU ARE CONSENTING TO BE BOUND BY AND ARE BECOMING A PARTY TO THIS AGREEMENT, THE TERMS AND CONDITIONS OF WHICH SHALL PREVAIL IN GOVERNING YOUR RIGHTS OF USE. BY CLICKING THE "BECOME A MEMBER" BUTTON, THE INDIVIDUAL OR ENTITY LICENSING THE PRODUCT ("YOU") IS CONSENTING TO BE BOUND BY AND IS BECOMING A PARTY TO THIS AGREEMENT. IF LICENSEE DOES NOT AGREE TO ALL OF THE TERMS OF THIS AGREEMENT, THE BUTTON INDICATING "BECOME A MEMBER" MUST NOT BE SELECTED, AND LICENSEE MUST NOT INSTALL OR USE THE SOFTWARE.
"VTC" refers to Virtual Training Company,
"You" refers to the user or subscriber.
"Software" refers to the VTC training content and software.
2. LICENSE: VTC hereby grants to You a worldwide, non-royalty bearing, non-exclusive license to use the Software according to the provisions contained herein and subject to payment of the applicable subscription fees.
3. RESTRICTIONS: You may not do any of the following:
Save the Software to Your hard disk or other storage
medium; permit others to use the Software except as specified by addendum;
modify, reverse engineer, decompile, or disassemble the Software; make
derivative works based on the Software; publish or otherwise disseminate
the Software. VTC, Inc., VTC Online University, and the Virtual Training
Company site is owned and operated by VTC, Inc. as a corporation of
All materials on this site are the property of VTC unless otherwise specified. No material from these pages may be copied, reproduced, republished, downloaded, uploaded, posted, transmitted, or distributed in any way. Modification of the materials or use of the materials for any other purpose is a violation of U.S. copyright law and other proprietary rights. For purposes of this Agreement, the use of any such material on any other web site or networked computer environment is prohibited.
4. FEES: The rights granted under this Agreement
are effective only upon payment of the subscription fees, which are
strictly non-refundable other than as expressly provided herein. The
term "monthly subscription" is defined as any 30 day period.
The term "yearly subscription" is defined as one 365 day
period. A yearly subscription ends on the same numerical date as it
began (example July 28, 2004 to July 28, 2005).
The VTC Online University is access to every VTC training tutorial in our library. You pay a flat fee for access to these titles. You are billed according to your renewal selection below, and can renew monthly, yearly, or in any other increment offered. If you choose to be billed monthly, you will be billed every 30 days for the subscription until you request the subscription be cancelled. Our terms of service state that you must cancel a monthly subscription at least two business days before your renewal date. These two days give us enough time to ensure that you will not be charged again.
5. LIMITED WARRANTY: VTC warrants that the Software, if operated as directed, will substantially achieve the functionality described. VTC does not warrant, however, that Your use of the Software will be uninterrupted or that the operation of the Software will be error-free or secure. In addition, the security mechanisms implemented by the Software have inherent limitations, and You must determine that the Software sufficiently meets Your requirements. VTC also warrants that the media containing the Software, if provided by VTC, is free from defects in material from the date You acquired the Software. VTC's sole liability for any breach of this warranty shall be, in VTC's sole discretion: (i) to replace Your defective media or Software; or (ii) to advise You how to achieve substantially the same functionality with the Software as described; or (iii) if the above remedies are impracticable, to refund the subscription fee You paid for the Software. Only if You inform VTC of Your problem with the Software during the applicable subscription period will VTC be obligated to honor this warranty. VTC will use reasonable commercial efforts to repair, replace, advise, or refund pursuant to the foregoing warranty within thirty (30) days of being so notified. If any modifications are made to the Software by You during the warranty period; if the medium is subjected to accident, abuse, or improper use; or if You violate the terms of this Agreement, then this warranty shall immediately terminate. This warranty shall not apply if the Software is used on or in conjunction with hardware or software other than the unmodified version of hardware and software with which the Software was designed to be used as described.
THIS IS A LIMITED WARRANTY, AND IT IS THE ONLY WARRANTY MADE BY VTC OR ITS SUPPLIERS. VTC MAKES NO OTHER WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT OF THIRD PARTIES' RIGHTS. YOU MAY HAVE OTHER STATUTORY RIGHTS. HOWEVER, TO THE FULL EXTENT PERMITTED BY LAW, THE DURATION OF STATUTORILY REQUIRED WARRANTIES, IF ANY, SHALL BE LIMITED TO THE ABOVE LIMITED WARRANTY PERIOD. MOREOVER, IN NO EVENT WILL WARRANTIES PROVIDED BY LAW, IF ANY, APPLY UNLESS THEY ARE REQUIRED TO APPLY BY STATUTE NOTWITHSTANDING THEIR EXCLUSION BY CONTRACT. NO DEALER, AGENT, OR EMPLOYEE OF VTC IS AUTHORIZED TO MAKE ANY MODIFICATIONS, EXTENSIONS, OR ADDITIONS TO THIS LIMITED WARRANTY.
6. PROPRIETARY RIGHTS: VTC reserves all proprietary rights in and to the Software, is protected by copyright and other intellectual property laws and by international treaties. VTC, Inc.
Trademark Notice: VTC, Virtual Training Company,
Inc., The VTC Logo, and VTC Online University, are trademarks of VTC,
Inc. All other company and product names may be trademarks of their
The information contained herein is subject to change without notice. Copyright © 1995 - 2005 VTC, Inc. All rights reserved.
7. TERMINATION: This Agreement shall automatically terminate if You fail to comply with the restrictions described herein. Your obligations to pay outstanding subscription fees shall survive any termination of this Agreement.
8. LIMITATION OF LIABILITY: UNDER NO CIRCUMSTANCES
AND UNDER NO LEGAL THEORY, TORT, CONTRACT, OR OTHERWISE, SHALL VTC
OR ITS SUPPLIERS OR RESELLERS BE LIABLE TO YOU OR ANY OTHER PERSON
FOR ANY INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES OF
ANY CHARACTER, INCLUDING WITHOUT LIMITATION, DAMAGES FOR LOSS OF GOODWILL,
WORK STOPPAGE, COMPUTER FAILURE OR MALFUNCTION, OR ANY AND ALL OTHER
COMMERCIAL DAMAGES OR LOSSES. IN NO EVENT WILL VTC BE LIABLE FOR ANY
DAMAGES IN EXCESS OF THE AMOUNT VTC RECEIVED FROM YOU FOR A LICENSE
TO THE SOFTWARE, EVEN IF VTC SHALL HAVE BEEN INFORMED OF THE POSSIBILITY
DAMAGES, OR FOR ANY CLAIM BY ANY OTHER PARTY. THIS LIMITATION OF LIABILITY SHALL NOT APPLY TO LIABILITY FOR DEATH OR PERSONAL INJURY RESULTING FROM VTC'S NEGLIGENCE TO THE EXTENT APPLICABLE LAW PROHIBITS SUCH LIMITATION. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OR LIMITATION OF INCIDENTAL OR CONSEQUENTIAL DAMAGES, SO THIS EXCLUSION AND LIMITATION MAY NOT APPLY TO YOU.
9. Links To Other Materials: Linked sites found at the VTC site are not under the control of VTC, and we are not responsible for the content of any linked site or any link contained in a linked site. VTC may change links based solely on our discretion, and we reserve the right to terminate any link or linking program at any time. VTC does not, by linking to sites, endorse companies or products to which it links and reserves the right to note as such on its web pages. If you decide to access any of the third party sites linked to this site, you do this entirely at your own risk.
Forums, and Chat are not always screened by VTC, and we are not responsible for the content of any public or open forum content at the site. VTC may change these public forums based solely on our discretion, and we reserve the right to terminate any forum at any time. VTC does not, by allowing these forums, endorse companies or products which may be mentioned in these forums, and reserves the right to note as such on its web pages. If you decide to access any of the public forums in this site, or linked to this site, you do this entirely at your own risk.
9. GOVERNING LAW & DISPUTE RESOLUTION: This Agreement is governed by Virginia law. All disputes between You and VTC shall be finally resolved through arbitration in Winchester, Virginia. This site is controlled by VTC from its offices within the United States of America. VTC makes no representation that materials in the site are appropriate or available for use in other locations, and access to them from territories where their content is illegal is prohibited. Those who choose to access this site from other locations do so on their own initiative and are responsible for compliance with applicable local laws. You may not use or export the Materials in violation of U.S. export laws and regulations. Any claim relating to the Materials shall be governed by the internal substantive laws of the Commonwealth of Virginia, USA.
VTC may revise these Terms at any time by updating this posting. You should visit this page from time to time to review the then-current Terms because they are binding on you. Certain provisions of these Terms may be superseded by expressly designated legal notices or terms located on particular pages at this Site.
If you have any questions regarding this policy,
or your information specifically,
you may email us at:firstname.lastname@example.org.