Top Description Fields Constructors Methods

org.jabref.model.ai.tokenization

public final Enum TokenEstimatorKind

Class Inheritance

extends Enum<TokenEstimatorKind>

java.lang.Object
java.lang.Enum
org.jabref.model.ai.tokenization.TokenEstimatorKind

Imports

org.jabref.model.ai.AiDefaultEnums

Idea taken from: https://community.openai.com/t/what-is-the-openai-algorithm-to-calculate-tokens/58237/4.

Field Summary

Modifier and Type	Field and Description
public static final TokenEstimatorKind	AVERAGE Average between `TokenEstimatorKind#WORDS` and `TokenEstimatorKind#CHARS`.
public static final TokenEstimatorKind	CHARS 4 characters = 1 token.
public static final TokenEstimatorKind	MAX Maximum between `TokenEstimatorKind#WORDS` and `TokenEstimatorKind#CHARS`.
public static final TokenEstimatorKind	MIN Minimum between `TokenEstimatorKind#WORDS` and `TokenEstimatorKind#CHARS`.
public static final TokenEstimatorKind	WORDS 0.75 words = 1 token.

Constructor Summary

Access	Constructor and Description
private	TokenEstimatorKind()

Method Summary

Modifier and Type	Method and Description
public static TokenEstimatorKind	safeValueOf(String name)
public static TokenEstimatorKind	valueOf(String name)
public static TokenEstimatorKind[]	values()

Inherited from java.lang.Enum:: clone compareTo describeConstable equals finalize getDeclaringClass hashCode name ordinal toString valueOf

Field Detail

AVERAGE	back to summary
public static final TokenEstimatorKind AVERAGE Average between `TokenEstimatorKind#WORDS` and `TokenEstimatorKind#CHARS`.

CHARS	back to summary
public static final TokenEstimatorKind CHARS 4 characters = 1 token.

MAX	back to summary
public static final TokenEstimatorKind MAX Maximum between `TokenEstimatorKind#WORDS` and `TokenEstimatorKind#CHARS`.

MIN	back to summary
public static final TokenEstimatorKind MIN Minimum between `TokenEstimatorKind#WORDS` and `TokenEstimatorKind#CHARS`.

WORDS	back to summary
public static final TokenEstimatorKind WORDS 0.75 words = 1 token.

Constructor Detail

TokenEstimatorKind	back to summary
private TokenEstimatorKind()

Method Detail

safeValueOf	back to summary
public static TokenEstimatorKind safeValueOf(String name)

valueOf	back to summary
public static TokenEstimatorKind valueOf(String name)

values	back to summary
public static TokenEstimatorKind[] values()