Verified Owner
Top Description Fields Constructors Methods
org.jabref.logic.database

public Class DuplicateCheck

extends Object
Class Inheritance
Imports
java.util.Collection, .HashMap, .HashSet, .Locale, .Map, .Objects, .Optional, .Set, java.util.regex.Pattern, java.util.stream.Collectors, org.jabref.logic.os.OS, org.jabref.logic.util.strings.StringSimilarity, .StringUtil, org.jabref.model.database.BibDatabase, .BibDatabaseMode, org.jabref.model.entry.AuthorList, .BibEntry, .BibEntryType, .BibEntryTypesManager, org.jabref.model.entry.field.BibField, .Field, .FieldProperty, .OrFields, .StandardField, org.jabref.model.entry.identifier.ISBN, org.jabref.model.entry.types.StandardEntryType, com.google.common.collect.Sets, org.slf4j.Logger, .LoggerFactory
This class contains utility method for duplicate checking of entries.

Field Summary

Modifier and TypeField and Description
private static final double
DOUBT_RANGE

private static final double
DUPLICATE_THRESHOLD

private static final int
EMPTY_IN_BOTH

private static final int
EMPTY_IN_ONE

private static final int
EMPTY_IN_TWO

private final BibEntryTypesManager
entryTypesManager

private static final int
EQUAL

private static final Map<Field, Double>
FIELD_WEIGHTS

private static final Logger
LOGGER

private static final int
NOT_EQUAL

private static final Pattern
PAGE_SEPARATOR_PATTERN

private static final double
REQUIRED_WEIGHT

private static final Set<StandardEntryType>
STANDARD_ENTRY_TYPES

Constructor Summary

AccessConstructor and Description
public
DuplicateCheck(BibEntryTypesManager entryTypesManager)

Method Summary

Modifier and TypeMethod and Description
private static int
compareAuthorField(final String stringOne, final String stringTwo)

private static int
compareChapterField(final String stringOne, final String stringTwo)

public static double
compareEntriesStrictly(BibEntry one, BibEntry two)

private static int
compareField(final String stringOne, final String stringTwo)

private static double[]
compareFieldSet(final Collection<Field> fields, final BibEntry one, final BibEntry two)

private static int
compareJournalField(final String stringOne, final String stringTwo)

We do not attempt to harmonize abbreviation state of the journal names, but we remove periods from the names in case they are abbreviated with and without dots:

private static boolean
compareOptionalFields(final BibEntryType type, final BibEntry one, final BibEntry two, final double[] req)

private static int
comparePagesField(final String stringOne, final String stringTwo)

Pages can be given with a variety of delimiters, "-", "--", " - ", " -- ".

private static double[]
compareRequiredFields(final BibEntryType type, final BibEntry one, final BibEntry two)

private static int
compareSingleField(final Field field, final BibEntry one, final BibEntry two)

public Optional<BibEntry>

Returns:

The first duplicate entry found. Empty Optional if no duplicates are found.
containsDuplicate(final BibDatabase
The database to search.
database, final BibEntry
The entry of which we are looking for duplicates.
entry, final BibDatabaseMode bibDatabaseMode)

Goes through all entries in the given database, and if at least one of them is a duplicate of the given entry, as per Util.isDuplicate(BibEntry, BibEntry), the duplicate is returned.

public Optional<BibEntry>

Returns:

The first duplicate entry found. Empty Optional if no duplicates are found.
containsDuplicate(final Collection<BibEntry>
The entries to search.
entries, final BibEntry
The entry of which we are looking for duplicates.
entry, final BibDatabaseMode bibDatabaseMode)

Same as containsDuplicate(BibDatabase, BibEntry, BibDatabaseMode), but searches a plain collection.

private static boolean
haveDifferentChaptersOrPagesOfTheSameBook(final BibEntry one, final BibEntry two)

private static boolean
haveDifferentEditions(final BibEntry one, final BibEntry two)

private static boolean
haveDifferentEntryType(final BibEntry one, final BibEntry two)

private static boolean
haveSameIdentifier(final BibEntry one, final BibEntry two)

private static boolean
isCommentEqual(BibEntry one, BibEntry two)

public boolean
isDuplicate(final BibEntry one, final BibEntry two, final BibDatabaseMode bibDatabaseMode)

Checks if the two entries represent the same publication.

private static boolean
isFarFromThreshold(double value)

private static boolean

Returns:

true if the content is equal (with normalized linebreaks), false otherwise.
isSingleFieldEqual(BibEntry one, BibEntry two, Field field)

Compares the string content of the given field at each entry character by character.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait

Field Detail

DOUBT_RANGEback to summary
private static final double DOUBT_RANGE
DUPLICATE_THRESHOLDback to summary
private static final double DUPLICATE_THRESHOLD
EMPTY_IN_BOTHback to summary
private static final int EMPTY_IN_BOTH
EMPTY_IN_ONEback to summary
private static final int EMPTY_IN_ONE
EMPTY_IN_TWOback to summary
private static final int EMPTY_IN_TWO
entryTypesManagerback to summary
private final BibEntryTypesManager entryTypesManager
EQUALback to summary
private static final int EQUAL
FIELD_WEIGHTSback to summary
private static final Map<Field, Double> FIELD_WEIGHTS
LOGGERback to summary
private static final Logger LOGGER
NOT_EQUALback to summary
private static final int NOT_EQUAL
PAGE_SEPARATOR_PATTERNback to summary
private static final Pattern PAGE_SEPARATOR_PATTERN
REQUIRED_WEIGHTback to summary
private static final double REQUIRED_WEIGHT
STANDARD_ENTRY_TYPESback to summary
private static final Set<StandardEntryType> STANDARD_ENTRY_TYPES

Constructor Detail

DuplicateCheckback to summary
public DuplicateCheck(BibEntryTypesManager entryTypesManager)

Method Detail

compareAuthorFieldback to summary
private static int compareAuthorField(final String stringOne, final String stringTwo)
compareChapterFieldback to summary
private static int compareChapterField(final String stringOne, final String stringTwo)
compareEntriesStrictlyback to summary
public static double compareEntriesStrictly(BibEntry one, BibEntry two)
compareFieldback to summary
private static int compareField(final String stringOne, final String stringTwo)
compareFieldSetback to summary
private static double[] compareFieldSet(final Collection<Field> fields, final BibEntry one, final BibEntry two)
compareJournalFieldback to summary
private static int compareJournalField(final String stringOne, final String stringTwo)

We do not attempt to harmonize abbreviation state of the journal names, but we remove periods from the names in case they are abbreviated with and without dots:

compareOptionalFieldsback to summary
private static boolean compareOptionalFields(final BibEntryType type, final BibEntry one, final BibEntry two, final double[] req)
comparePagesFieldback to summary
private static int comparePagesField(final String stringOne, final String stringTwo)

Pages can be given with a variety of delimiters, "-", "--", " - ", " -- ". We do a replace to harmonize these to a simple "-" After this, a simple test for equality should be enough

compareRequiredFieldsback to summary
private static double[] compareRequiredFields(final BibEntryType type, final BibEntry one, final BibEntry two)
compareSingleFieldback to summary
private static int compareSingleField(final Field field, final BibEntry one, final BibEntry two)
containsDuplicateback to summary
public Optional<BibEntry> containsDuplicate(final BibDatabase database, final BibEntry entry, final BibDatabaseMode bibDatabaseMode)

Goes through all entries in the given database, and if at least one of them is a duplicate of the given entry, as per Util.isDuplicate(BibEntry, BibEntry), the duplicate is returned. The search is terminated when the first duplicate is found.

Parameters
database:BibDatabase

The database to search.

entry:BibEntry

The entry of which we are looking for duplicates.

Returns:Optional<BibEntry>

The first duplicate entry found. Empty Optional if no duplicates are found.

containsDuplicateback to summary
public Optional<BibEntry> containsDuplicate(final Collection<BibEntry> entries, final BibEntry entry, final BibDatabaseMode bibDatabaseMode)

Same as containsDuplicate(BibDatabase, BibEntry, BibDatabaseMode), but searches a plain collection.

Callers which do not run on the JavaFX Application Thread should pass a snapshot: BibDatabase#getEntries returns a live view of the entry list, which may be modified while it is iterated.

Parameters
entries:Collection<BibEntry>

The entries to search.

entry:BibEntry

The entry of which we are looking for duplicates.

Returns:Optional<BibEntry>

The first duplicate entry found. Empty Optional if no duplicates are found.

haveDifferentChaptersOrPagesOfTheSameBookback to summary
private static boolean haveDifferentChaptersOrPagesOfTheSameBook(final BibEntry one, final BibEntry two)
haveDifferentEditionsback to summary
private static boolean haveDifferentEditions(final BibEntry one, final BibEntry two)
haveDifferentEntryTypeback to summary
private static boolean haveDifferentEntryType(final BibEntry one, final BibEntry two)
haveSameIdentifierback to summary
private static boolean haveSameIdentifier(final BibEntry one, final BibEntry two)
isCommentEqualback to summary
private static boolean isCommentEqual(BibEntry one, BibEntry two)
isDuplicateback to summary
public boolean isDuplicate(final BibEntry one, final BibEntry two, final BibDatabaseMode bibDatabaseMode)

Checks if the two entries represent the same publication.

isFarFromThresholdback to summary
private static boolean isFarFromThreshold(double value)
isSingleFieldEqualback to summary
private static boolean isSingleFieldEqual(BibEntry one, BibEntry two, Field field)

Compares the string content of the given field at each entry character by character.

Returns:boolean

true if the content is equal (with normalized linebreaks), false otherwise.