DUFF(1)

NAME
duff - duplicate file finder

SYNOPSIS
duff [-HLPaeqprtz] [-f format] [-l limit] [file ...]
duff [-h]
duff [-v]

DESCRIPTION
The duff utility reports any clusters of duplicates in the specified files and/or directories. For each cluster of duplicate files, duff prints a customizable header on one line, and then the names of all the files in the cluster, each on one line.

If no files are specified as arguments, duff reads file names from stdin.

Note that as of version 0.4, duff ignores symbolic links to files, as that behavior was conceptually broken. Therefore, the -H, -L and -P options now apply only to directories.

The following options are available:

-H  Follow symbolic links listed on the command line. This overrides any previous -L or -P option. Note that this only applies to directories, as symbolic links to files are never followed.

-L  Follow all symbolic links. This overrides any previous -H or -P option. Note that this only applies to directories, as symbolic links to files are never followed.

-P  Don't follow any symbolic links. This overrides any previous -H or -L option. This is the default. Note that this only applies to directories, as symbolic links to files are never followed.

-a  Include hidden files and directories when searching recursively.

-e  Excess mode. List all excess files; i.e. all but one of each cluster of duplicates. This is useful when you want to automate removal of duplicate files.

-f format
    Set the format of the cluster header. If the header is set to the empty string, no header line is printed.

    The following escape sequences are available:

    %n  The number of files in the cluster.

    %c  The SHA1 checksum of files in the cluster.

    %i  The one-based index of the file cluster.

    %s  The size, in bytes, of a file in the cluster.

    %%  A ‘%’ character.

    The default format string is:

        %n files in cluster %i (%s bytes, checksum %c)

-h  Display help information and exit.

-l limit
    The minimum size of files to be sampled. If the size of files in a cluster is equal or greater than the specified limit, duff will sample a few bytes from each file, before calculating a full checksum. This is an optimization for larger files. The default limit is 1048576 bytes.

-q  Quiet mode. Supress warnings and error messages.

-p  Physical mode. Make duff consider physical files instead of hard links. If specified, multiple hard links to the same physical file will not be reported as duplicates.

-r  Recursively search into all specified directories.

-t  Thorough mode. Distrust checksums as a guarantee for equality. In thorough mode, duff compares files byte by byte when their checksums match.

-v  Display version information and exit.

-z  Do not consider files with zero size to be equal. This option suppresses reporting a cluster of empty files.

EXAMPLES
The command:
    duff -e * | xargs rm

removes all duplicate files in the current directory.

The command:
    duff -r foo/

lists all duplicate files in the directory foo and its subdirectories.

The command:
    find. -name '*.h' | duff

lists all duplicate header files in the current directory and its subdirectories.

DIAGNOSTICS

SEE ALSO
find(1), xargs(1)

AUTHORS
Camilla Berglund <elmindreda@users.sourceforge.net>

BUGS
duff no longer considers symbolic links to files potential duplicates, and will not report them. This may break scripts relying on the previous behavior.