Display/Remove duplicate bookmarks or empty folders
# bookmarkdupes
(C) Martin Väth <[email protected]>, <[email protected]>
This project is under the GNU Public License 2.0.
A WebExtension which can display/remove duplicate bookmarks or empty folders.
After installing bookmarkdupes, the usage is rather simple:
To open bookmarkdupes, click the extension symbol (duplicate stars) or
use the link in the options page of the extension.
**Do not enable “Expert Mode”** (unless you fully understand the consequences,
see separate section).
Then select what you want to display:
1. Bookmark duplicates
2. Empty folders
3. Non-duplicate bookmarks
After this, you will be offered the list of bookmarks with checkboxes;
in case 1 the numbers indicate the order in which matching bookmarks
were added according to the internally stored date.
There are also buttons to select/unselect convenient sets of checkboxes.
Finally, there are buttons to remove the selected bookmarks.
**Be aware that removing bookmarks is irreversible!**
*It is recommended to make backups of your bookmarks first!*
Currently, there is no working version of the extension available for android
(see section **Known Bugs**).
## Important
When you reorganized/added/removed bookmarks, make sure to update the
displayed list (by pressing the corresponding button) before removing
bookmarks!
## Permissions
The extension requires the following permissions for these reasons:
1. “bookmarks” to read/modify bookmarks
2. “storage” to store/restore the customized rules in expert mode.
Unfortunately, “storage” is not one of the optional permissions which might
be required only if that feature is actually requested by the user.
## Expert Mode
When selecting the checkbox for expert mode, details can be configured to
ignore certain bookmarks when calculating the list or under which cases
bookmarks are considered to be dupes of each other.
In non-expert mode, two bookmarks are considered to be duplicates of each
other if their URL coincides.
In expert mode there are custom rules by which the URL which is actually
used for comparison is to be modified. The details are as follows.
For every bookmark the rules are applied in the given order.
There are 2 types of rules: Filter rules and URL modification rules
(there are also disabled rules which are only listed but have no effect).
For both types of rules 4 regular expressions can be specified which are used
to determine if the rule applies: If the corresponding regular expression is
nonempty, the corresponding condition must be satisfied or the rule will not
apply. (In the case of filter rules, at least one of these 4 regular
expressions must be nonempty or that filter rules will not apply either.)
The 4 regular expressions refer to the bookmark's name or url, respectively,
and the regular expressions must either match or not match, respectively.
- The term “regular expression” refers to a javascript type regular expression.
- The bookmark's name refers to the full bookmark path as it appears in the
browser with folder names separated by the null character.
For instance, if you have in “Bookmark Menu” a folder “Collection”
which contains your bookmark "Example", the bookmark's name becomes
`Bookmark Menu\0Collection\0Example` (where `\0` denotes the null character
which can be matched by the regular expression with `\0` or `\x00`).
- The bookmark's URL refers to the bookmark URL after possible modifications
by previous modifier rules.
If a filter rule applies, the corresponding bookmark is ignored, i.e.,
it will not be considered as a duplicate and will neither appear in the
list of empty folders nor of all bookmarks.
If a URL modification rule applies, a text replacement will occur:
All parts matching a specified regular expression are substituted by a
replacement text (which might be empty).
The rules for this correspond to the javascript String.prototype.replace
function with the global modifier.
In particular, the replacement text can contain symbols like
`$&` or `$1` to refer to the whole matched text or to the
match of the first brace in the regular expression, respectively.
The following 5 replacement texts have a special meaning which goes beyond
the standard javascript replacement rules.
Note that this special meaning is only active if this is the full replacement
text. In other words, to get the special meaning, the replacement text must not
contain anything else than these 4-6 characters.
(This does not really restrict the functionality, because one can combine
several rules to work around that limitation if necessary.)
1. `\L$&` is the match in lower case.
2. `\U$&` is the match in upper case.
3. `$URL` is the url before any other rules have been applied.
4. `$NAME` is the bookmark's full name/path with folders separated by the
null character, e.g. “Bookmark Menu\0Collection\0Example”
5. `$TITLE` is the bookmark's title without the path; for instance, for the
bookmark name “Bookmark Menu | Collection | Example” it is “Example”.
## Examples for special tasks which can be done in Expert Mode
### Task
1. Consider two bookmarks as duplicate if their URL differs only in `https:` vs. `http:` at the beginnning.
2. Consider two bookmarks as duplicate if their URL matches up to the first `?` symbol, i.e. only their additional information differs.
3. Do not consider any bookmark from any folder named ``Mr. Dupe`` as a duplicate.
4. Add all bookmarks to the list of duplicates, whether duplicate or not.
5. Add all bookmarks from the folder `Bookmark Menu | Remove` to the list of duplicates (no matter whether they actually are duplicates).
6. Consider two bookmarks as duplicate if they have the same name (instead of the same URL).
7. Consider two bookmarks as duplicate if they reside in the same folder.
### How to do the above tasks in Expert Mode
1. Use the replacement rule: “Replace URL matches” `^http:` “by” `^https:`
Explanation: By replacing everywhere the URL beginnning with `http:`
by `https:`, it does not matter whether the actual URL started with `http:`
or `https:`. Of course, we could have exchanged the roles of `http`
and `https` in our rule.
2. Use the replacement rule: “Replace URL matches” `\?.*` “by” ` ` (empty string)
Explanation: Simply omit the ? symbol and all other symbols following it
in all URLs.
3. Use the filter rule: “Name matches” `\0Mr\. Dupe\0` or `\0Mr[.] Dupe\0`
Explanation: If a bookmark is in a folder named “Mr. Dupe”, its full name (path) will contain the text “\0Mr. Dupe\0”; so match that text. Since the “.” symbol has a special meaning for regular expressions, we have to quote it. This can be done by either `\.` or by looking for a character class `[…]` which contains only the single symbol `.`.
4. Use the replacement rule: “Replace URL matches” `.+` “by” `constant`
Explanation: Pretend that every bookmark has the URL `constant` by replacing all characters (`.+`) of the original URL by that text.
5. Use the replacement rule: “Name Matches” `^Bookmark Menu\0Remove\0` “Replace URL matches” `.+` “by” `constant`
As in 4, but only for bookmarks whose full name starts with the matching path.
This works only if the folder contains at least 2 bookmarks (because otherwise `constant` is not a duplicate URL). Of course, one might use an actually duplicate URL instead of `constant` to work around this limitation.
6. Use the replacement rule: “Replace URL matches” `.+` “by” `$TITLE`
Explanation: Replace all bookmark URLs by the corresponding bookmark title when looking for dupes.
7. Use the replacement rule “Replace URL matches” `.+` “by” `$NAME` followed by a further replacement rule “Replace URL matches” `[^\0]*$` “by” ` ` (empty string)
Explanation: First replace the URL by its full name path, and then omit the last component of this path by cutting of the longest sequence of non-`\0`-symbols at the end.
## Contributors
(in alphabetical order)
- Essam Ambr aka (JaMaYcKa) (Arabic translation)
- Henaro aka Ironwool (Russian and Ukrainian translation; redesign icon in svg; provide favicon)
- Sopor (Swedish translation)
- Juan Salvador Aleixandre Talens (Spanish translation)
- YFdyh000 (Simplified Chinese translation)
Latest reviews
- (2023-11-01) Hambone Designs: Does exactly what you think it should. Love this.
- (2023-10-26) 郭奇异: you saves my life!
- (2023-10-24) Helar Kallas: Thanks!!! Deleted 45K of duplicates from 20+ years with 0 effort. Tried several other tools but they were rather pointless, like having to select each bookmark manually?! :D
- (2023-10-11) fruck wheeled: Allowed me to find and conveniently move NON duplicate bookmarks from several imported bookmark folders, fast easy awesome compared to the un-intuitive and low capability competition. Thanks.
- (2023-09-26) Wesley Moy: Exactly what I needed. The extension provided all the options I needed to clean up my duplicates surprisingly quickly and with a clear understanding what I'm doing.
- (2023-09-09) الخبراء Experts: there are several apps but the automation is what I downloaded for it is a fantastic feature
- (2023-09-08) Nam Tran: Absolutely excellent and straightforward. I tried two other bookmark dupe removers, but it did not work perfectly like this once. :D
- (2023-07-20) Gen G: Excellent!
- (2023-06-06) Sandré de Paula: Perfeito, maravilhoso! eu tinha mais de 3800 favoritos (bookmarks) e com a ajuda dele eliminei os duplicados, eliminei as pastas vazias e agora tenho só 900 favoritos! Resolveu a bagunça de duplicados!
- (2023-05-30) Mark Andrew Gerads: I tried 2 other bookmark dupe removers, and neither were as good as this one. I quite like the ability to remove all but the oldest instance of a duplicate, which is quite useful when one has hundreds of duplicates, which one would have when migrating between browsers.
- (2023-05-12) Peace Maker: I tried similar kind of two more extensions but none were perfect as this. The mass delete and marking folder features are awesome!! Thank so much : )
- (2023-03-27) Chuck Baggett ChuckBaggettWeb (ChuckBaggett web): It doesn't delete empty folders.
- (2023-03-26) LUCA BERTI: Fast Easy, top.
- (2023-03-12) Alex Mathew: Very intuitive, and super fast with the clean up.
- (2023-03-11) Dylan Morganti: awsome jsut awsome the name is just a small part of what this can do!
- (2023-02-06) Marcellea Levone: Worked great!!!
- (2023-02-02) Деметрий: Быстро нашёл и удалил лишнее)))
- (2023-01-16) Michael Ludlum: Did exactly what I needed and what I expected. It groups the duplicated bookmarks, let's me select all but 1 of the duplicates, and removes them for me. Not sure why the "Bookmarks cleanup" extension has so many more users. Great job, and thank-you for saving me a ton of time. Can I suggest that you find a why to get on the google results list for a query like "remove duplicate bookmarks chrome"?
- (2023-01-06) Mehdi: The best in its Kind!
- (2022-12-28) Pat MySecret: Fast and Efficient for those with a BIG need to cleanup It's awesome for duplicates and empties.. It has an advanced sorting feature, but could also use an additional simplified "find similar" feature. After you are finished with this extension which is the best of them, do yourself one more favor and use the extension "Fast Bookmark Scanner" to merge duplicate folders.
- (2022-11-07) Juandavid Ortiz: Muy fácil de usar, puede borrar todos los duplicados de manera rápida y eficiente. Además de eso también las carpetas vacías (yo lo hice como 4 veces porque tenía muchas carpetas dentro de carpetas jejejeje) Úsenlo cuando exporten sus favoritos o algo parecido.
- (2022-10-28) Nina Reviews: Essa é sem dúvida a melhor extensão para isso! Conheço ela a anos no Firefox, e fiquei Feliz em encontrá-la aqui também para o Chrome! Parabéns aos Desenvolvedores!
- (2022-10-12) E. O.: Tl;dr: yet another happy user! With this extension I managed broken sync of some tens of thousands bookmarks. The extension did the deduplication of bookmarks and empty folders exact and carefully, always displaying results before processing and getting a feedback from user before doing any critical action.
- (2022-09-16) Curtis Anderson: Overall a helpful extension and a good job by the dev. It does what it says and has some useful ways to handle duplicates and empty folders in bulk. The UI is bare-bones and doesn't quite project that it delivers a "clean" experience, but to install and run it, I had no errors. Would be nice to bundle this with features like scanning for dead links and timeouts. If it had a complete bookmark utility set of features and a UI upgrade, I'd give this 5 stars.
- (2022-08-31) robin allan: Magic!
- (2022-08-19) Ethan Newman: perfectly done job and easy to use
- (2022-05-12) Nabio N: Did the job as expected
- (2022-04-28) Андрей Ворошилов: Супер! Удалил все дубли и пустые папки в несколько кликов!
- (2022-04-24) Andrei Bintintan: Simple and good
- (2022-02-22) ayunami 2000: awesome works best extension and cool and good and i have nothing negative to say or add
- (2022-02-11) qiang bao: very useful!!!!
- (2022-02-08) Paul Kang: 试用了好几个重复书签管理插件,这个是最好用的
- (2022-01-07) Autumn Brown: I had over 240K bookmarks. Yep. I have no idea how that happened, but it did. After several days of consciously deduping, I now have 353. I know I lost a few because I don't really know how it works, but, I don't care. I wasn't using them anyway because they were such a mess. I love that I have 353 bookmarks that I can find and maybe, now that my bookmarks make sense again, I'll love creating new ones. Thank you so very much!
- (2021-12-16) bittersweet tea: Superb, made with efficiency in mind 10/10
- (2021-10-14) Ho Bac NGUYEN: thanks for very simple but effective tool
- (2021-10-06) Cenon Francis Manansala (Cee): The most reliable way to display/remove duplicate bookmarks or empty folders. I definitely recommend this extension to everyone!
- (2021-10-04) هاشم شرف الدين: اضافة ممتازة لإزالة علامات التبويب المكررة البرنامج يحتاج تركيز شوية لأن الخيارات كلها بالانجليزي
- (2021-09-30) WTC 911: اضافة عظيمه ومميزه
- (2021-08-28) Сергей: Всё отлично! Прекрасная работа расширения! Разработчикам спасибо!
- (2021-08-27) La Su: Quick, I try with others but this one performs very quick. And also it allow a selection per group of folder (nearly, but not as perfect as, like the app Dupplicate Cleaner Pro) which I have not seen many other extensions. The only thing it misses is Checking the validity of the bookmark, which I have to use other extension in parallel.
- (2021-08-26) Evie Hartman: Whoa. I had thousands and THOUSANDS of bookmarks spanning over 10 years, imported from other browsers, often with like 20 copies - it was insanity. Finally, finally, there is only one "disorganized bookmarks" folder left. And it only took a few rounds, using just the basic functions. All told, I'm pretty sure it took me less than 10 minutes. Everything about the basic functionality just makes sense and works. (I don't know if I'll ever really need the fancy ones - I don't how they work at all - but that's okay.) And I love that you can clearly see where exactly each bookmark exists in the hierarchy. I'm pretty sure this is the only Chrome extension I've ever rated. Thanks, mvath.develop!
- (2021-08-19) Sigit Arif Fianto: Thank alot its work like a charm
- (2021-08-09) Stephanie Lehrer: Awesome, simple app. Something went wrong with my chrome bookmark sync and it was duplicating tons of my bookmarks so this was an easy way to fix the problem and cleanup. thank you so much :)
- (2021-07-15) Tech 4242: its Perfectly Handy and Appreciations to Developber....
- (2021-05-28) Махмуд Фарух: Просто и быстро. Избавился от мусора - дублей и пустых папок. До этого пытался чистить вручную. Надоело. Решил попробовать расширение. Нашёл 1400 дублей. Руками бы удалял до морковкина заговенья. А с Bookmark Dupes - за минуту. Разработчику - респект и благодарность!
- (2021-05-26) Renato Portugal: Muito bom, remove os duplicados mesmo!!! Se melhorar a parte gráfica, teria 10 estrelas. Parabéns.
- (2021-05-13) Jesai Tarun: W.O.W! Thanks a ton for making this extension! :D I was about to import thousands of bookmarks into raindrop.io, but its free tier doesn't support duplicate and dead link finding - this was a life saver and got my thousands down by more than half haha! Tqsm dev! :D Works really great! :)
- (2021-05-05) Jigar Dave: Best . removed 1102 useless dupes. Does all work which you need related to bookmark cleaning
- (2021-04-28) 小ae6623: it is very useful !
- (2021-04-09) Data Storageson: Using this extension the way you want to, might have a considerable learning curve to it, as you might want to edit/add some expert mode rules (luckily you can read some info of this app's Overview). E.g. to match the Bookmarks by their name, you need to have a Modify rule (one of the three preset buttons), where you add the following search criteria: ".+" (without " "-s) in the [Replace URL matches] box and "$TITLE" in the [by] box. Then you might want to save the settings to your local storage. On the plus side: it's the most powerful tool for finding duplicate bookmarks I've yet stumbled upon. Hence my high rating. Things that could improve (don't take this critique close to heart and sorry for the wall of text): 1. A help/readme feature in the extension itself would be nice. Had no a user's manual - found a description in the Overview section on this here extension's website. 2. A more lamer-friendly (intuitive) language use in the 'expert mode'. People who have little to no experience with scripting/coding may not grasp the needed concepts, and relying on their intuition, might end up being confused. We – the lamers – think in: "I want Dupes to find all the bookmarks with duplicate names". If not instructed, we're not expecting to change the thinking to something like: "I want Dupes to use the 'Modify' rule with the regular expression 'Replace URL matches' set to value [.+] and the regular expression 'by' set to value [$TITLE]". (Pardon me if it was a gibberish!) Here's what I mean: 1) the 'Name' is/isn't box pair refers to the folders where the bookmarks are located in (in other words: path), not the bookmarks' names – as I guessed at first glance, I admit. 2) with the 'URL' is/isn't rule-box pair it might be somewhat easier – one is where, URL must include something you want to (depends on the rule ->) filter out & the other is the opposite. 3) the wording of the third box pair's title ('Replace URL matches' & 'by') might cause a confusion of the highest order. At first I thought it's a feature that would change duplicate bookmarks' URLs – which didn't make sense, considering this extension is not about editing bookmarks. Turned out instead it lets me "put a mask onto the compared parts" – either replacing, or concealing them. I.e. Dupes doesn't care what it compares, meaning we can modify things it compares – the whole URLs (".+") – by changing what it sees (hence the masking analogy) to – let's say – Bookmarks' names ("$TITLE"). This will force Dupes to consider bookmarks as duplicates if they have the same name (instead of the same URL). We can also hide parts of the URLs that we want to be ignored. Dupes will only see everything before the question mark ("\?.*") as if that symbol & everything that follows weren't there (" " – empty box). This allows for ignoring such parts of URLs which send requests to the servers (e.g. this here, gathering info for Google Analytics or some such: "?utm_medium=Social&utm_source=Twitter[...]").