For those who have a suspicion they’ve downloaded the same file to several directories but can’t be bothered with a manual search, a Linux duplicate file finder is the much-needed solution. Just having a program like that in your system will help you keep it organized and unencumbered. Finding decent Linux programs is pretty hard but good thing you don’t need to do a thorough search yourself, just pick one of the options from below.
Top 5 Linux Duplicate File Finders
- FSLint - Simple GUI software
- dupeGuru - User-friendly interface
- Fdupes - Byte-by-byte verification
- Rdfind - Ranking algorithm
- Speedy Duplicate Finder - Cross-folders search
Any system needs to stay organized, and today it’s harder than ever to keep an eye on that. Whether it is for a Linux computer or server, there are efficient and reliable ways to remove duplicate files. You can set some command-line tools to the task or use the GUI tools instead. Whichever way you prefer, there is a good program for that below.
1. FSLint - Our Choice
- Substitutes duplicates with a hard link
- Detects problematic file names
- Creates find-format library paths
- None
Verdict: This free utility can be used on a range of OS distributions and varying software repositories which include Ubuntu as well. It can be employed as a duplicate photo finder but there are other helpful tasks it can carry out. For instance, the program can locate empty directories, temporary files, symbolic links and even conflicting names, to name just a few things.
The installation can be launched with this command:
- sudo apt install fslint
To set the Linux duplicate file finder going, add the path of whatever directory you want to scan into the Search bar and click Find at the bottom left of the window. There is a helpful feature that recursively searches in directories and sub-directories, and you can tick it when necessary.
When the program has finished, you will see a list of all the findings and decide what to do with each. You can opt to save or delete the files, merge two copies or replace them with a Symlink.
It can be launched in either command-line or GUI mode.
2. dupeGuru
- Recognizes a multitude of extensions
- Image juxtaposition
- Helpful Ignore List, Auto-Select, 1-Click Removal features
- Inefficient image preview
Verdict: This popular Linux find duplicates program is actually cross-platform so could be used on other systems, too. It’s great for working with frequently used image extensions like JPG, PNG, even GIFs, but it can also help to organize your music, books and many other kinds of files. It can even pull out the files with similar names if you’re uncertain in your search request. The fuzzy matching algorithm is super-fast and speedily scans the folders. If you’re ready to test it out, use these commands:
- $ sudo add-apt-repository ppa:dupeguru/ppa
- $ sudo apt-get update
- $ sudo apt-get install dupeguru
Now you can launch the program either through the Unity Dash or using the Menu. At the bottom of the workspace, you will find a “+” button that is used to add directories for the new search. Once you’ve added all, click Scan and wait for the results to appear. If there is anything, the Ubuntu duplicate file finder will surely locate it. Then, you can go through the list and decide what to do with each result. You can remove the extra copy, rename it or move elsewhere.
Verdict: This free program is open-source, and the C programming language was used for its code. It offers several modes of searching, such as by size, full or partial MD5 signatures and by comparing each bite. If you would like to test this option, just follow these steps to launch the installation from official repositories:
- $ sudo pacman -S fdupes
- $ sudo apt-get install fdupes
- $ sudo dnf install fdupes
On Linux remove duplicate files by indicating the directory path following the program’s name in the command line: fdupes/home/chris. This would result in the list of repeating files in this particular directory only, not its subdirectories. If you want to include the latter, add “-r” right after the program’s name to start a recursive search through everything that directory contains. The program will make no changes to the files it finds so you can do that by hand where you see the need.
4. Rdfind
- Easy in use
- Allows filtering duplicate groups
- Accelerated search mode
- Performs file listing through database instead of RAM
Verdict: This is a free Ubuntu find duplicate files program that can ransack multiple directories. It has a special algorithm that can compare not just the names of files but their contents too. Another algorithm can actually detect the original among the duplicates in case you want to preserve that specifically. If this sounds like what you need, follow the simple installation procedure:
- $ sudo apt-get install rdfind
- $ sudo dnf install rdfind
- $ sudo pacman -S rdfind
Now, you can command your new Linux duplicate file finder to search a particular directory. This is one of the simpler programs that doesn’t allow any manipulations so as an outcome you will get a text file listing every detected duplicate. It will be saved to the working directory and you can manually perform any organizing steps if you find them necessary.
5. Speedy Duplicate Finder
- Powerful program for speedy searches
- Quite straightforward
- Allows canceling actions
- Stuffy interface
Verdict: This is not just a duplicate detection but a data recovery software as well. There is a free version available but you will have to be content with only the basic tools at your disposal.
The program can be installed through the Ubuntu Software Center if you are working with this distribution. Whichever version you get, you will find it is a capable and quick program that’s easy to operate. Here is how you can launch the Linux duplicate file remover installation:
- $ sudo snap install speedy-duplicate-finder
First, add the folder you want to have looked through. Then, indicate what method you prefer applied to it. Finish by adding a filter if you require one for this task and then launch the scanning. When you get the list of findings, you can assign an action to each file, either deleting or moving it, and then perform it in bulk by pressing the button.
