Verified Owner
Top Description Inners Fields Constructors Methods
org.jabref.logic.importer.fetcher

public Class ArXivFetcher

extends Object
implements FulltextFetcher, PagedSearchBasedFetcher, IdBasedFetcher, IdFetcher<ArXivIdentifier>
Class Inheritance
All Implemented Interfaces
org.jabref.logic.importer.IdFetcher, org.jabref.logic.importer.WebFetcher, org.jabref.logic.importer.IdBasedFetcher, org.jabref.logic.importer.PagedSearchBasedFetcher, org.jabref.logic.importer.SearchBasedFetcher, org.jabref.logic.importer.FulltextFetcher
Imports
java.io.IOException, java.net.HttpURLConnection, .MalformedURLException, .URISyntaxException, .URL, java.time.Duration, java.util.ArrayList, .Collection, .List, .Map, .Optional, .Set, java.util.concurrent.CompletableFuture, .CompletionException, .ExecutorService, .Executors, java.util.regex.Pattern, java.util.stream.Collectors, javax.xml.parsers.DocumentBuilder, .DocumentBuilderFactory, .ParserConfigurationException, org.jabref.logic.cleanup.EprintCleanup, org.jabref.logic.help.HelpFile, org.jabref.logic.importer.FetcherClientException, .FetcherException, .FulltextFetcher, .IdBasedFetcher, .IdFetcher, .ImportFormatPreferences, .PagedSearchBasedFetcher, org.jabref.logic.importer.fetcher.transformers.ArXivQueryTransformer, org.jabref.logic.integrity.BracesCorrector, org.jabref.logic.util.URLUtil, org.jabref.logic.util.io.XMLUtil, org.jabref.logic.util.strings.StringSimilarity, .StringUtil, org.jabref.model.entry.BibEntry, .KeywordList, .LinkedFile, org.jabref.model.entry.field.Field, .InternalField, .StandardField, org.jabref.model.entry.identifier.ArXivIdentifier, .DOI, org.jabref.model.entry.types.StandardEntryType, org.jabref.model.paging.Page, org.jabref.model.search.query.BaseQueryNode, .SearchQueryNode, org.jabref.model.util.OptionalUtil, org.apache.hc.core5.net.URIBuilder, org.jspecify.annotations.NonNull, org.slf4j.Logger, .LoggerFactory, org.w3c.dom.Document, .Node, org.xml.sax.SAXException

Fetcher for ArXiv that merges fields from arXiv-issued DOIs (and user-issued ones when applicable) to get more information overall.

These are the post-processing steps applied to the original fetch from ArXiv's API:

  1. Use ArXiv-issued DOI to get more merge more data with original entry, overwriting some of those fields;
  2. Use user-issued DOI (if it was provided) to merge even more data with the result of the previous step, overwriting some of those fields;
  3. Modify keywords: remove repetitions and adapt some edge cases (commas in keyword transformed into forward slashes).
See Also
arXiv.org blog for more info about arXiv-issued DOIs, ArXiv API for an overview of the API, ArXiv API User's Manual for a detailed description on how to use the API

Nested and Inner Type Summary

Modifier and TypeClass and Description
protected static class
ArXivFetcher.ArXiv

Fetcher for the arXiv.

Field Summary

Modifier and TypeField and Description
private final ArXivFetcher.ArXiv
private static final Map<String, String>
private static final Set<Field>
private static final Set<Field>
private static final String
private final DoiFetcher
private final ImportFormatPreferences
private static final Pattern
private static final Logger
private static final Pattern
private static final Pattern
private static final Pattern

Constructor Summary

AccessConstructor and Description
public
ArXivFetcher(ImportFormatPreferences importFormatPreferences)

public
ArXivFetcher(ImportFormatPreferences importFormatPreferences, DoiFetcher
The fetcher, maybe be NULL if no additional search is desired.
doiFetcher
)

Method Summary

Modifier and TypeMethod and Description
private void
adaptKeywordsFrom(BibEntry
A BibEntry to modify
bibEntry
)

Remove duplicate values on "KEYWORD" field, if any.

private Page<BibEntry>
enrichPageWithDoi(Page<BibEntry> result)

Enriches each entry in the page with additional metadata from ArXiv-issued and user-issued DOIs using parallel async requests.

public Optional<URL>
public Optional<ArXivIdentifier>
findIdentifier(BibEntry entry)

Implements org.jabref.logic.importer.IdFetcher.findIdentifier.

Looks for an identifier based on the information stored in the given BibEntry.

private static String

Returns:

ArXiv-issued DOI
getAutomaticDoi
(String
An ArXiv ID
arXivId
)

Get ArXiv-issued DOI from the entry's arXiv ID

ArXiv-issued DOIs are identifiers associated with every ArXiv entry.

private static Optional<String>

Returns:

ArXiv-issued DOI, or Empty, if method could not retrieve it
getAutomaticDoi
(BibEntry
A Bibtex Entry, formatted as a ArXiv entry. Must contain an EPRINT field
arXivBibEntry
)

Get ArXiv-issued DOI from the arXiv entry itself.

private static String

Returns:

ArXiv-issued DOI
getAutomaticDoi
(ArXivIdentifier
An ArXiv ID as internal object
arXivId
)

Get ArXiv-issued DOI from ArXiv Identifier object

ArXiv-issued DOIs are identifiers associated with every ArXiv entry.

public Optional<HelpFile>
getHelpPage()

Overrides default org.jabref.logic.importer.WebFetcher.getHelpPage.

Returns the help page for this fetcher.

public String
getIdentifierName()

Implements org.jabref.logic.importer.IdFetcher.getIdentifierName.

Returns the name of the identifier that is returned by this fetcher.

private static Optional<String>

Returns:

User-issued DOI, if any field exists and if it's not an automatic one (see getAutomaticDoi(ArXivIdentifier))
getManualDoi
(BibEntry
An ArXiv Bibtex entry from where the DOI is extracted
arXivBibEntry
)

Get user-issued DOI from ArXiv Bibtex entry, if any

User-issued DOIs are identifiers associated with some ArXiv entries that can associate an entry with an external service, like Springer Link.

public String
getName()

Implements org.jabref.logic.importer.WebFetcher.getName.

Returns the localized name of this fetcher.

public TrustLevel
getTrustLevel()

Implements org.jabref.logic.importer.FulltextFetcher.getTrustLevel.

Returns the level of trust for this fetcher.

private void
infuseWithInspireCitationKeyIfMissing(BibEntry
The fetched arXiv entry to infuse, modified in place.
entry
)

If the entry has no "real" citation key yet, try to look up the paper on INSPIRE (the standard literature database for high-energy physics) and adopt its curated texkey instead.

private void
inplaceAsyncInfuseArXivWithDoi(BibEntry
An existing entry to be updated with new/modified fields
arXivBibEntry
)

Infuse arXivBibEntryPromise with additional fields in an asynchronous way

private void
inplaceAsyncInfuseArXivWithDoi(CompletableFuture<Optional<BibEntry>>
A future entry that (if it exists) will be updated with new/modified fields
arXivBibEntryFuture
,
Optional<ArXivIdentifier>
An ArXiv ID for the main reference (from ArXiv), so that the retrieval of ArXiv-issued DOI metadata can be faster
arXivId
)

Infuse arXivBibEntryFuture with additional fields in an asynchronous way, accelerating the process by providing a valid ArXiv ID

private static boolean
isManualDoi(String doi)

Check if a specific DOI is user-assigned.

private static boolean
private void
private void
mergeArXivEntryWithFutureDoiEntry(BibEntry
The entry to merge into
arXivEntry
,
CompletableFuture<Optional<BibEntry>>
A future result of the fetching process
bibEntryFuture
,
Set<Field>
Which fields from "bibEntryFuture" to prioritize, replacing them on "arXivEntry"
priorityFields
,
String
Identifier used in initiating the "bibEntryFuture" future (for logging). This is usually a DOI, but can be anything.
id
)

Eventually merge the ArXiv Bibtex entry with a Future Bibtex entry (ArXiv/user-assigned DOIs)

private static String
public Page<BibEntry>
public List<BibEntry>
performSearch(String searchQuery)

Overrides default org.jabref.logic.importer.SearchBasedFetcher.performSearch.

Looks for hits which are matched by the given free-text query.

public Optional<BibEntry>
performSearchById(String identifier)

Implements org.jabref.logic.importer.IdBasedFetcher.performSearchById.

Looks for bibliographic information associated to the given identifier.

public Page<BibEntry>
public Page<BibEntry>

Returns:

A list of entries matching the complex query
performSearchPaged
(BaseQueryNode
the first search query node
queryNode
,
int pageNumber)

Implements org.jabref.logic.importer.PagedSearchBasedFetcher.performSearchPaged.

Constructs a complex query string using the field prefixes specified at https://arxiv.org/help/api/user-manual and modify resulting BibEntries with additional info from the ArXiv-issued DOI

private static Optional<BibEntry>

Returns:

the fetch result
waitForBibEntryRetrieval
(CompletableFuture<Optional<BibEntry>>
A CompletableFuture that parallelize the API fetching process
bibEntryFuture
)

Get the Bibtex Entry from a Future API request (uses blocking) and treat exceptions.

Inherited from java.lang.Object:
cloneequalsfinalizegetClasshashCodenotifynotifyAlltoStringwaitwaitwait