Verified Owner
Top Description Inners Fields Constructors Methods
org.jabref.logic.importer.fileformat.pdf

public Class PdfContentImporter

extends PdfImporter
Class Inheritance
Imports
java.io.IOException, .StringWriter, java.nio.file.Path, java.util.ArrayList, .Collections, .Comparator, .HashMap, .List, .Locale, .Map, .Optional, .TreeMap, java.util.regex.Matcher, .Pattern, org.jabref.logic.importer.ParserResult, org.jabref.logic.l10n.Localization, org.jabref.logic.os.OS, org.jabref.logic.util.PdfUtils, org.jabref.logic.util.strings.StringUtil, org.jabref.model.entry.BibEntry, org.jabref.model.entry.field.StandardField, org.jabref.model.entry.identifier.ArXivIdentifier, .DOI, org.jabref.model.entry.types.EntryType, .StandardEntryType, com.google.common.annotations.VisibleForTesting, com.google.common.base.Strings, org.apache.pdfbox.pdmodel.PDDocument, org.apache.pdfbox.text.PDFTextStripper, .TextPosition, org.jspecify.annotations.NullMarked, .Nullable

Parses data of the first page of the PDF and creates a BibTeX entry.

Currently, Springer, and IEEE formats are supported.

In case one wants to have a list of BibEntry matching the bibliography of a PDF, please see RuleBasedBibliographyPdfImporter.

If several PDF importers should be tried, use PdfMergeMetadataImporter.

Nested and Inner Type Summary

Modifier and TypeClass and Description
private static class

Field Summary

Modifier and TypeField and Description
private static final int
private String
private int
private String[]
private String
private static final Pattern

Constructor Summary

AccessConstructor and Description
public

Method Summary

Modifier and TypeMethod and Description
private static Optional<String>
private void
extractYear()

Extract the year out of curString (if it is not yet defined)

private void
fillCurStringWithNonEmptyLines()

Fill curString with lines until "" is found No trailing space is added i is advanced to the next non-empty line (ignoring white space)

Lines containing only white spaces are ignored, but NOT considered as ""

Uses GLOBAL variables lines, curLine, i

private String
getArXivId(String arXivId)

public String
getDescription()

Implements abstract org.jabref.logic.importer.Importer.getDescription.

Returns the description of the import format.

private @Nullable String
getDoi(@Nullable String currentDoi)

pack-priv Optional<BibEntry>

Returns:

An Optional containing a BibEntry with the parsed bibliographic data if extraction is successful. Otherwise, an empty Optional.
getEntryFromPDFContent
(String
The raw content of the PDF's first page, which may contain metadata and main content.
firstpageContents
,
String
The line separator used to format and unify line breaks in the text content.
lineSeparator
,
Optional<String>
An optional title string determined by font size; if provided, this overrides the default title parsing.
titleByFontSize
)

Parses the first page content of a PDF document and extracts bibliographic information such as title, author, abstract, keywords, and other relevant metadata.

public String
getId()

Implements abstract org.jabref.logic.importer.Importer.getId.

Returns a slug, which identifies this importer.

public String
getName()

Implements abstract org.jabref.logic.importer.Importer.getName.

Returns the name of this import format.

public ParserResult
importDatabase(Path filePath, PDDocument document)

Implements abstract org.jabref.logic.importer.fileformat.pdf.PdfImporter.importDatabase.

Note

The "throws" clause is for "developer convenience" - it is turned into a ParserResult at importDatabase(Path).

private void
proceedToNextNonEmptyLine()

PDFTextStripper normally does NOT produce multiple empty lines (besides at strange PDFs).

private void
readLastBlock()

resets curString curString now contains the last block (until "" reached) Trailing space is added

invariant before/after: i points to line before the last handled block

private String
removeNonLettersAtEnd(String input)

Removes all non-letter characters at the end

EXCEPTION: a closing bracket is NOT removed

Todo

Additionally replace multiple subsequent spaces by one space, which will cause a rename of this method

private String
streamlineNames(String names)

Converts an author line as found in a PDF (a "byline") into a BibTeX author list, mapping a trailing et al. to others.

private String
Inherited from org.jabref.logic.importer.fileformat.pdf.PdfImporter:
getFileTypeimportDatabaseimportDatabaseimportDatabaseisRecognizedFormatisRecognizedFormat

Field Detail

ARXIV_PREFIX_LENGTHback to summary
private static final int ARXIV_PREFIX_LENGTH
curStringback to summary
private String curString
lineIndexback to summary
private int lineIndex
linesback to summary
private String[] lines
yearback to summary
private String year
YEAR_EXTRACT_PATTERNback to summary
private static final Pattern YEAR_EXTRACT_PATTERN

Constructor Detail

PdfContentImporterback to summary
public PdfContentImporter()

Method Detail

extractTitleFromDocumentback to summary
private static Optional<String> extractTitleFromDocument(PDDocument document) throws IOException
extractYearback to summary
private void extractYear()

Extract the year out of curString (if it is not yet defined)

fillCurStringWithNonEmptyLinesback to summary
private void fillCurStringWithNonEmptyLines()

Fill curString with lines until "" is found No trailing space is added i is advanced to the next non-empty line (ignoring white space)

Lines containing only white spaces are ignored, but NOT considered as ""

Uses GLOBAL variables lines, curLine, i

getArXivIdback to summary
private String getArXivId(String arXivId)
getDescriptionback to summary
public String getDescription()

Implements abstract org.jabref.logic.importer.Importer.getDescription.

Doc from org.jabref.logic.importer.Importer.getDescription.

Returns the description of the import format.

The description should specify
what kind of entries from what sources and based on what specification it is able to import

by what criteria it isRecognizedFormat(BufferedReader) recognizes an import format

Annotations
@Override
getDoiback to summary
private @Nullable String getDoi(@Nullable String currentDoi)
getEntryFromPDFContentback to summary
pack-priv Optional<BibEntry> getEntryFromPDFContent(String firstpageContents, String lineSeparator, Optional<String> titleByFontSize)

Parses the first page content of a PDF document and extracts bibliographic information such as title, author, abstract, keywords, and other relevant metadata. This method processes the content line-by-line and uses custom parsing logic to identify and assemble information blocks from academic papers.

idea: split[] contains the different lines, blocks are separated by empty lines, treat each block or do special treatment at authors (which are not broken). Therefore, we do a line-based and not a block-based splitting i points to the current line curString (mostly) contains the current block, the different lines are joined into one and thereby separated by " "

This method follows the structure typically found in academic paper PDFs:

  • First, it attempts to detect the title by font size, if available, or by text position.
  • Authors are then processed line-by-line until reaching the next section.
  • Abstract and keywords, if found, are extracted as they appear on the page.
  • Finally, conference details, DOI, and publication information are parsed from the lower blocks.

The parsing logic also identifies and categorizes entries based on keywords such as "Abstract" or "Keywords" and specific terms that denote sections. Additionally, this method can handle publisher-specific formats like Springer or IEEE, extracting data like series, volume, and conference titles.

Parameters
firstpageContents:String

The raw content of the PDF's first page, which may contain metadata and main content.

lineSeparator:String

The line separator used to format and unify line breaks in the text content.

titleByFontSize:Optional<String>

An optional title string determined by font size; if provided, this overrides the default title parsing.

Returns:Optional<BibEntry>

An Optional containing a BibEntry with the parsed bibliographic data if extraction is successful. Otherwise, an empty Optional.

Annotations
@VisibleForTesting
getIdback to summary
public String getId()

Implements abstract org.jabref.logic.importer.Importer.getId.

Doc from org.jabref.logic.importer.Importer.getId.

Returns a slug, which identifies this importer. Used, for example, to identify the importer in CLI.

Typically, this name should be short, in English, and should not contain special characters like #, %, etc.

Annotations
@Override
getNameback to summary
public String getName()

Implements abstract org.jabref.logic.importer.Importer.getName.

Doc from org.jabref.logic.importer.Importer.getName.

Returns the name of this import format. Typically, this is a string that denotes file type or format. It can also be localized, like "XMP-annotated PDF".

Annotations
@Override
importDatabaseback to summary
public ParserResult importDatabase(Path filePath, PDDocument document) throws IOException

Implements abstract org.jabref.logic.importer.fileformat.pdf.PdfImporter.importDatabase.

Doc from org.jabref.logic.importer.fileformat.pdf.PdfImporter.importDatabase.

Note

The "throws" clause is for "developer convenience" - it is turned into a ParserResult at importDatabase(Path).

Annotations
@Override
proceedToNextNonEmptyLineback to summary
private void proceedToNextNonEmptyLine()

PDFTextStripper normally does NOT produce multiple empty lines (besides at strange PDFs). These strange PDFs are handled here: proceed to next non-empty line

readLastBlockback to summary
private void readLastBlock()

resets curString curString now contains the last block (until "" reached) Trailing space is added

invariant before/after: i points to line before the last handled block

removeNonLettersAtEndback to summary
private String removeNonLettersAtEnd(String input)

Removes all non-letter characters at the end

EXCEPTION: a closing bracket is NOT removed

Todo

Additionally replace multiple subsequent spaces by one space, which will cause a rename of this method

streamlineNamesback to summary
private String streamlineNames(String names)

Converts an author line as found in a PDF (a "byline") into a BibTeX author list, mapping a trailing et al. to others.

Neither org.jabref.logic.importer.AuthorListParser nor org.jabref.logic.formatter.bibtexfields.NormalizeNamesFormatter (which delegates to org.jabref.logic.importer.AuthorListParser#normalizeSimply) can take the byline directly. Both keep BibTeX semantics, where a comma separates family and given names: Smith, John and Doe, Jane has the same shape as Karen Cooper, Jennifer Donovan and Gary Williamson. Only this importer knows that its input is a byline and thus in "Given Family" order. Measured on the raw bylines of the importer's tests:

Byline AuthorListParser result
Karen A. Cooper, Jennifer L. Donovan, Andrew L. Waterhouse and Gary Williamson 2 persons: "Andrew L. Waterhouse Karen A. Cooper, Jennifer L. Donovan" and "Gary Williamson"
Karen A. Cooper1, Jennifer L. Donovan2 and Gary Williamson1* 2 persons, affiliation markers kept as part of the family names
Karen A. Cooper, Jennifer L. Donovan, et al. 1 garbled person plus others (the initials-first heuristic of normalizeSimply needs K. A. Cooper)
Anke Lüdeling Merja Kytö (separated by spaces only) 1 person

Hence the byline is split heuristically here; the result is a plain BibTeX list that org.jabref.model.entry.AuthorList#parse handles afterwards.

streamlineTitleback to summary
private String streamlineTitle(String title)
org.jabref.logic.importer.fileformat.pdf back to summary

private Class PdfContentImporter.TitleExtractorByFontSize

extends PDFTextStripper
Class Inheritance

Nested and Inner Type Summary

Modifier and TypeClass and Description
private static record

Field Summary

Modifier and TypeField and Description
private final List<List<TextPosition>>
Inherited from org.apache.pdfbox.text.PDFTextStripper:
charactersByArticledocumentLINE_SEPARATORoutput

Constructor Summary

AccessConstructor and Description
public

Method Summary

Modifier and TypeMethod and Description
private void
private Optional<String>
public Optional<String>
getTitle(PDDocument document)

private boolean
isFarAway(TextPosition previous, TextPosition current)

private boolean
isLegalTitle(String candidateText)

private boolean
private boolean
isUnwantedText(TextPosition previousTextPosition, TextPosition textPosition, Map<Float, TextPosition> lastPositionMap, float fontSize)

protected void
writeString(String text, List<TextPosition> textPositions)

Overrides org.apache.pdfbox.text.PDFTextStripper.writeString.

Write a Java string to the output stream.

Inherited from org.apache.pdfbox.text.PDFTextStripper:
beginMarkedContentSequenceendArticleendDocumentendMarkedContentSequenceendPagegetAddMoreFormattinggetArticleEndgetArticleStartgetAverageCharTolerancegetCharactersByArticlegetCurrentPageNogetDropThresholdgetEndBookmarkgetEndPagegetIgnoreContentStreamSpaceGlyphsgetIndentThresholdgetLineSeparatorgetListItemPatternsgetOutputgetPageEndgetPageStartgetParagraphEndgetParagraphStartgetSeparateByBeadsgetSortByPositiongetSpacingTolerancegetStartBookmarkgetStartPagegetSuppressDuplicateOverlappingTextgetTextgetWordSeparatormatchPatternprocessPageprocessPagesprocessTextPositionsetAddMoreFormattingsetArticleEndsetArticleStartsetAverageCharTolerancesetDropThresholdsetEndBookmarksetEndPagesetIgnoreContentStreamSpaceGlyphssetIndentThresholdsetLineSeparatorsetListItemPatternssetPageEndsetPageStartsetParagraphEndsetParagraphStartsetShouldSeparateByBeadssetSortByPositionsetSpacingTolerancesetStartBookmarksetStartPagesetSuppressDuplicateOverlappingTextsetWordSeparatorstartArticlestartArticlestartDocumentstartPagewriteCharacterswriteLineSeparatorwritePagewritePageEndwritePageStartwriteParagraphEndwriteParagraphSeparatorwriteParagraphStartwriteStringwriteTextwriteWordSeparator

Field Detail

textPositionsPerPageback to summary
private final List<List<TextPosition>> textPositionsPerPage

Constructor Detail

TitleExtractorByFontSizeback to summary
public TitleExtractorByFontSize()

Method Detail

collectCandidatesback to summary
private void collectCandidates(List<TextPosition> textPositions, int pageIndex, List<PdfContentImporter.TitleExtractorByFontSize.TitleCandidate> candidates)
findLargestFontTextback to summary
private Optional<String> findLargestFontText()
getTitleback to summary
public Optional<String> getTitle(PDDocument document) throws IOException
isFarAwayback to summary
private boolean isFarAway(TextPosition previous, TextPosition current)
isLegalTitleback to summary
private boolean isLegalTitle(String candidateText)
isThereSpaceback to summary
private boolean isThereSpace(TextPosition previous, TextPosition current)
isUnwantedTextback to summary
private boolean isUnwantedText(TextPosition previousTextPosition, TextPosition textPosition, Map<Float, TextPosition> lastPositionMap, float fontSize)
writeStringback to summary
protected void writeString(String text, List<TextPosition> textPositions)

Overrides org.apache.pdfbox.text.PDFTextStripper.writeString.

Doc from org.apache.pdfbox.text.PDFTextStripper.writeString.

Write a Java string to the output stream. The default implementation will ignore the textPositions and just calls writeString(String).

Annotations
@Override
org.jabref.logic.importer.fileformat.pdf back to summary

private final Record PdfContentImporter.TitleExtractorByFontSize.TitleCandidate

extends Record
Class Inheritance
Annotations
@NullMarked
Record Components
fontSize:float 
pageIndex:int 
text:String 

Field Summary

Modifier and TypeField and Description
private final float
fontSize

Record Component accessed by fontSize().

private final int
pageIndex

Record Component accessed by pageIndex().

private final String
text

Record Component accessed by text().

Constructor Summary

AccessConstructor and Description
private
TitleCandidate(float fontSize, int pageIndex, String text)

Method Summary

Modifier and TypeMethod and Description
public final boolean
equals(Object o)

Implements abstract java.lang.Record.equals.

Indicates whether some other object is "equal to" this one.

public float
fontSize()

Record Component getter of fontSize.

public final int
hashCode()

Implements abstract java.lang.Record.hashCode.

Returns a hash code value for the record.

public int
pageIndex()

Record Component getter of pageIndex.

public String
text()

Record Component getter of text.

public final String
toString()

Implements abstract java.lang.Record.toString.

Returns a string representation of the record.

Field Detail

fontSizeback to summary
private final float fontSize

Record Component accessed by fontSize().

pageIndexback to summary
private final int pageIndex

Record Component accessed by pageIndex().

textback to summary
private final String text

Record Component accessed by text().

Constructor Detail

TitleCandidateback to summary
private TitleCandidate(float fontSize, int pageIndex, String text)

Method Detail

equalsback to summary
public final boolean equals(Object o)

Implements abstract java.lang.Record.equals.

Doc from java.lang.Record.equals.

Indicates whether some other object is "equal to" this one. In addition to the general contract of Object.equals, record classes must further obey the invariant that when a record instance is "copied" by passing the result of the record component accessor methods to the canonical constructor, as follows:

    R copy = new R(r.c1(), r.c2(), ..., r.cn());
then it must be the case that r.equals(copy).
fontSizeback to summary
public float fontSize()

Record Component getter of fontSize.

hashCodeback to summary
public final int hashCode()

Implements abstract java.lang.Record.hashCode.

Doc from java.lang.Record.hashCode.

Returns a hash code value for the record. Obeys the general contract of Object.hashCode. For records, hashing behavior is constrained by the refined contract of Record.equals, so that any two records created from the same components must have the same hash code.

pageIndexback to summary
public int pageIndex()

Record Component getter of pageIndex.

textback to summary
public String text()

Record Component getter of text.

toStringback to summary
public final String toString()

Implements abstract java.lang.Record.toString.

Doc from java.lang.Record.toString.

Returns a string representation of the record. In accordance with the general contract of Object#toString(), the toString method returns a string that "textually represents" this record. The result should be a concise but informative representation that is easy for a person to read.

In addition to this general contract, record classes must further participate in the invariant that any two records which are equal must produce equal strings. This invariant is necessarily relaxed in the rare case where corresponding equal component values might fail to produce equal strings for themselves.