Download Ancestry DNA Match Information
Ancestry Match Downloader is designed to use the new Ancestry DNA API to scan, store, and download your current matches. The extension limits the download to matches having 20cm or more in common. This the 4th Cousin or Closer filter that Ancestry uses. Ancestry does not include matches with less than 20cm shared in their in common matching. Typically a user will have from up to 10000 matches that fall in this range. The match scan will take approximately 1 minute per 5000 matches. Scanning in common matches still takes a long time to complete. If you have 2500 matches expect ICW scan to take around 30 minutes. If you have 4000 matches expect it to take 2 hours. If you have 6000 matches expect it to take 3 to 4 hours. Updates: Version 0.1.4 Changed the default filenames for export to include the test takers name. Also I included the filter settings for Clusters in the file name. The Cluster file name will be Name-ICW-Clusters-MaxCM-MinCM-Percent-MinSize.XML. Version 0.1.3 Added message box to export routines to show number of matches exported and clusters exported. Added more error catching to the exports. Version 0.1.2 Added some error catching to the export routines to catch incomplete scans of match data. Added an message pop up when match scan is complete and ICW scan is complete. Added an alert message if communication to Ancestry server is interrupted during a scan. Version 0.1.1 Eliminated current ICW matches label. When scanning ICW matches if you have close family matches it may take up to 15 seconds or so for the scanned ICW matches number to increase. Be patient and trust that it is working. Or open the developer tools with Ctrl+Shift+I and click the network tab to view the network requests in real time. Added tree information to the cluster output. The column beside the matches name will tell you if there is a common ancestor identified, public or private tree, and number of people in the tree if it is linked to the match. If there is no tree public or private it will simply say no tree. Version 0.1.0 Cleaned up the user interface. Added a filter to not include single match clusters. Set minimum cluster size to 2 or more to filter small clusters. Version 0.0.9 Fixed some bugs that could cause corrupt data to get saved to storage. Combined the scan match information and tree information to a single button press. Earlier Versions Fixed a bug in the filter for clusters. Fixed a bug that prevented the last few ICW matches from being saved. Modified saving of match data to storage to reduce cpu load. Modified the export icw matrix to reduce memory usage. Large numbers of matches were causing out of memory crashes. Changed extension to load into a new tab by default instead of being an iframe. Scans now continue if you change to a new tab or window. Scan times greatly reduced. Modified the In common matches download to eliminate unneeded requests. This should speed up ICW downloads some. Fixed a bug in storing In Common with matches Updated match scanning for updates to Ancestry API that increased matches returned per page from 20 to 100. This greatly reduces the time to scan matches and ICW matches. Scans should now complete 5x faster than previously. To Do: I am working on downloading matches pedigree trees and combining them into a single gedcom file. Currently Ancestry beta allows viewing a 6 generation pedigree tree of matches even if you are not subscribed to ancestry's service. Instructions for use: In order for the extension to work you must be signed in to your Ancestry account where you have access to the DNA test. I suggest opening a new window in chrome and then browsing to Ancestry.com. Make sure you are logged in and then click the Extension ICON. The extension will load in a new tab and will stay active even if you switch to other tabs or windows. 1: On first use of the extension click the "Get Test IDs" button to populate the drop down with the available DNA tests on the signed in account. The IDs will be stored locally. You will only need to click the "Get Test IDs" the first time you run the extension unless you add more testers to your account. 2: Select the Tester's name from the drop down selection box. This will populate the Number of matches field with the amount of matches currently available to scan for this tester. Note: If you have previously scanned this tester it will also populate the number of matches previously scanned, number of matches scanned for tree information, and the number of matches that have been scanned for "In Common With" matches. 3: To begin scanning the matches click the "Scan matches" button. If you have previously scanned your matches a prompt will appear to verify you wish to rescan the matches. "Match information scanned" will update as the matches are scanned and works as a progress indicator. After all matches are scanned it will then scan the matches for tree information. "Tree information scanned" will update to show progress of the scan. Once complete Match information and Tree information numbers should match the "Number of Matches". Note: Ancestry's new api does not provide a means of getting only new matches so each time you wish to update the match database you must rescan all of your matches. Note: The scan does not change the viewed status of a match. It essentially creates a list equivalent to the list of matches seen on Ancestry's site. Note: If internet connection is lost or Ancestry's server has problems during the scan and the number of matches is different from the match information and tree information scanned you will need to rescan the matches by pressing scan matches and selecting OK from the message box. Note: The program will retry a scan 3 times prior to giving up. 4. The Matches in common with scan is a long scan but is required to use the clustering tool. The initial scan time will depend greatly on the number of matches you have and how interrelated those matches are. Note: A tester that has 2000 matches where the average matches in common for each match is less than 100 will take 30 mins to complete. Note: A tester that has 5000 matches where the average matches in common for each match is 200 will take 3 hours to complete. 5. If somehow the Matches in common scan fails (internet connection fails etc..) the last match fully scanned will be saved. You can resume the scan as long as you have not run the Scan Matches to update your list of scanned matches. Click the resume button to continue a failed scan of ICW matches. Note: Clicking Resume will prompt you to continue a failed scan. Select ok to continue. 6. If you have updated your scanned matches since last running "Scan ICW" select "Update" instead of "Resume". This will continue scanning your in common matches and will update the previously scanned matches that are in common with any new matches. 7. "Export Matches" button will dump all the saved match information for the current tester in CSV format. Note: ICW match IDs are appended at the end of each row of matches. Note: The columns labeled 1000 to 1023 represent any custom color tags that the tester has used to tag their matches. Note: Export may take up to 1 minute to complete if you have a large number of matches. 8. "Export" button creates 2 CSV files. The first containing the match information without the ICW data. The second being a matrix of the tester's matches with the ICW matches marked with an "X". Note: The 2nd file ICW Matrix may take up to 1 min to output. 9. "Cluster" button runs a clustering routine on the ICW match data. It creates clusters based on the matches that fall between "Max CM" and "Min CM". Use the "Min percent of matches in common" to tighten up the clusters. Default is 50% however 65% or more may give better results. The Max CM should be set below the value of any 1C1R matches for best results. The min CM will increase or decrease the number of clusters. Set minimum cluster size to 2 or more to filter small clusters. Note: The output of the Clusters is an Excel XML Spreadsheet file. This format can be directly opened in Excel 2007 or newer. Note: The XML will have 1 worksheet for each cluster and will already have the rows/columns formatted to make viewing easier. Note: The routine counts the number of times a match shows up as an ICW match to those in the cluster and the value shown in each column is that. Note: The matches are sorted based on that value. Note: I hope to continue working on the algorithm and improve the cluster outputs in future versions. Note: If a CSV file output is desired I can add this to the output. All of the clusters would have to be on a single sheet though as CSV does not support multiple sheets.
- (2019-11-17) Masterboy Haron: Why it takes only matches with more than 20cM ? And how to take all matches?
- (2019-11-08) Maria Mueller: I receive a message that my cluster files are corrupted. They will not open in Execl.
- (2019-10-20) L Forbes: Definitely a really nice tool! Thank you and keep up the good work.
- (2019-10-19) mike dack: Great app/extension thank you! I really missed Ancestry DNA Helper when Ancestry updated their presentation of results with new features such as Thrulines. This app fills the bill very nicely and even grabs the 'user added notes', thanks.
- (2019-10-07) Mary Brandenburg: This is incredible! So useful to have an "ICW Matrix," to be able to change the thresholds... it's just great! If you could add an html file with a colored, visual representation of the clusters, then I would totally pay for this extension ;-) Thank you for creating such a great tool!
- (2019-08-15) Eric Cotter: Nice set of tools! Keep it coming!
- (2019-08-14) Barbara Taylor: Download files have a generic name "Select a Test".... not easy to see which person is being downloaded.
- (2019-08-12) Walter Bays: At a minimum it lets you get to all the match information which recent Ancestry search changes have made so difficult to find otherwise. At best the clusters it identifies can focus your search to break brick walls. It doesn't handle large data sets so well, producing XML files too large for LibreOffice to load. Maybe Excel can load them. So I wrote my own clustering scripts using the CSV files this extension provided.
- (2020-01-16, v:0.1.4) Terri Stern: Error Message: There was an error communicating with Ancestry Server try your scan again
Today I'm getting this message. It was working fine yesterday. Did something change at Ancestry?
- (2020-01-06, v:0.1.4) Jenny Irwin: Error with creating a cluster file
Hello, Thank you for creating this extension! I was able to export my match data and match data with ICW matrix. However, when attempting to create clusters and export XML (I changed min percent to 65 and kept all other settings as default), I received the message "Found 15 clusters and exported 6 filtered clusters" but when I opened the file I received the following error message: The XML file Jenny Rizzo Irwin-ICW clusters 400-40-65-2.xml can not be opened because there is no available data view (XSLT).
- (2019-12-15, v:0.1.4) Mary McCampbell Bell: Column headings
This tool worked fine for me; however, could you explain what the various columns are? Like True or False etc. Thank you! Mary M. Bell
- (2019-11-21, v:0.1.4) Scott D: Download family surnames as well?
Would it be possible to download a list of surnames for each DNA match? I've developed an algorithm to look for names that are statistically over-represented in various groups, however there is no easy way to actually get these lists of names. The point is to be able to better determine how matches are related to each other and what a likely common ancestor might be.
- (2019-10-03, v:0.1.4) harold watson: is there an explanation of the column headings available.
i am not clear on Relationship Group. also are the steps that allow anyone to expand the scan to 10 cM?
- (2019-09-17, v:0.1.4) Mark Hunnibell: How to get deeper than 20 cM (down to 6)?
I see you stop at 20 cM but I need to go deeper for work on my project. AncestryDNA has some kind of technical problem getting me past 10 cM using the "scroll down" to get the pages... either Chrome crashes "aw snap" or I get a message saying the service is "overtaxed". How can I get past 20 cM?
- (2019-09-12, v:0.1.4) Carol Andrews: Is this extension still working with Ancestry.co.uk?
Receive error message "There was an error communicating with Ancestry Server try your scan again".
- (2019-09-08, v:0.1.4) Extend match list
Due to the changes made by Ancestry on July 1, 2019, the DNA Helper Extension no longer functions. There are no current plans to make DNA Helper Extension compatible with the new version of Ancestry. So it would be a great help to be able to get a simplified list of ALL matches. Most of my Common Ancestor matches occur below the 20cM threshold yet even the 6cM ones include Shared Matches. OK the matrix would be impossibly big, but the list would allow sharing and cross matching using SQL. David
- (2019-09-05, v:0.1.4) Robert Hungerschafer: Export Match Data and ICW matrix Enhancement
On ICW matrix, if you replaced X's with 1's, then you could quickly sum by column and row and incorporate weighting. Bob
- (2019-08-13, v:0.1.4) David Aarons: "Error communicating with the Ancestry server" message
I'm a UK-based Ancestry user and get the above message when I try to"Get test IDs". Can you help please?
- (2019-08-03, v:0.1.4) Julia Vollmers: ideas for your work in ancestry dna
hi - would love to brainstorm with an idea of mine - not exactly what you're doing, but maybe you can help me work it out
- (2019-06-14, v:0.1.2) Amanda Carter: Github
Hey, is this project on github?