This class, based on Unicode Technical Report #36 and Unicode Technical Standard #39, has two main functions:

1. Checking whether two strings are visually confusable with each other, such as "desparejado" and "ԁеѕрагејаԁо".
2. Checking whether an individual string is likely to be an attempt at confusing the reader (spoof detection), such as "pаypаl" spelled with Cyrillic 'а' characters.

Although originally designed as a method for flagging suspicious identifier strings such as URLs, SpoofChecker has a number of other practical use cases, such as preventing attempts to evade bad-word content filters.

Confusables

The following example shows how to use SpoofChecker to check for confusability between two strings:

SpoofChecker sc = new SpoofChecker.Builder().setChecks(SpoofChecker.CONFUSABLE).build();
int result = sc.areConfusable("desparejado", "ԁеѕрагејаԁо");
System.out.println(result != 0);  // true

SpoofChecker uses a builder paradigm: options are specified within the context of a lightweight SpoofChecker.Builder object, and upon calling SpoofChecker.Builder#build, expensive data loading operations are performed, and an immutable SpoofChecker is returned.

The first line of the example creates a SpoofChecker object with confusable-checking enabled; the second line performs the confusability test. For best performance, the instance should be created once (e.g., upon application startup), and the more efficient SpoofChecker#areConfusable method can be used at runtime.

If the paragraph direction used to display the strings is known, it should be passed to SpoofChecker#areConfusable:

// These strings look identical when rendered in a left-to-right context.
// They look distinct in a right-to-left context.
String s1 = "A1א";  // A1א
String s2 = "Aא1";  // Aא1

SpoofChecker sc = new SpoofChecker.Builder().setChecks(SpoofChecker.CONFUSABLE).build();
int result = sc.areConfusable(Bidi.DIRECTION_LEFT_TO_RIGHT, s1, s2);
System.out.println(result != 0);  // true

UTS 39 defines two strings to be confusable if they map to the same skeleton. A skeleton is a sequence of families of confusable characters, where each family has a single exemplar character. SpoofChecker#getSkeleton computes the skeleton for a particular string, so the following snippet is equivalent to the example above:

SpoofChecker sc = new SpoofChecker.Builder().setChecks(SpoofChecker.CONFUSABLE).build();
boolean result = sc.getSkeleton("desparejado").equals(sc.getSkeleton("ԁеѕрагејаԁо"));
System.out.println(result);  // true

If you need to check if a string is confusable with any string in a dictionary of many strings, rather than calling SpoofChecker#areConfusable many times in a loop, SpoofChecker#getSkeleton can be used instead, as shown below:

// Setup:
String[] DICTIONARY = new String[]{ "lorem", "ipsum" }; // example
SpoofChecker sc = new SpoofChecker.Builder().setChecks(SpoofChecker.CONFUSABLE).build();
HashSet<String> skeletons = new HashSet<String>();
for (String word : DICTIONARY) {
  skeletons.add(sc.getSkeleton(word));
}

// Live Check:
boolean result = skeletons.contains(sc.getSkeleton("1orern"));
System.out.println(result);  // true

Spoof Detection

The following snippet shows a minimal example of using SpoofChecker to perform spoof detection on a string:

SpoofChecker sc = new SpoofChecker.Builder()
    .setAllowedChars(SpoofChecker.RECOMMENDED.cloneAsThawed().addAll(SpoofChecker.INCLUSION))
    .setRestrictionLevel(SpoofChecker.RestrictionLevel.MODERATELY_RESTRICTIVE)
    .setChecks(SpoofChecker.ALL_CHECKS &~ SpoofChecker.CONFUSABLE)
    .build();
boolean result = sc.failsChecks("pаypаl");  // with Cyrillic 'а' characters
System.out.println(result);  // true

As in the case for confusability checking, it is good practice to create one SpoofChecker instance at startup, and call the cheaper SpoofChecker#failsChecks online. In the second line, we specify the set of allowed characters to be those with type RECOMMENDED or INCLUSION, according to the recommendation in UTS 39. In the third line, the CONFUSABLE checks are disabled. It is good practice to disable them if you won't be using the instance to perform confusability checking.

To get more details on why a string failed the checks, use a SpoofChecker.CheckResult:

SpoofChecker sc = new SpoofChecker.Builder()
    .setAllowedChars(SpoofChecker.RECOMMENDED.cloneAsThawed().addAll(SpoofChecker.INCLUSION))
    .setRestrictionLevel(SpoofChecker.RestrictionLevel.MODERATELY_RESTRICTIVE)
    .setChecks(SpoofChecker.ALL_CHECKS &~ SpoofChecker.CONFUSABLE)
    .build();
SpoofChecker.CheckResult checkResult = new SpoofChecker.CheckResult();
boolean result = sc.failsChecks("pаypаl", checkResult);
System.out.println(checkResult.checks);  // 16

The return value is a bitmask of the checks that failed. In this case, there was one check that failed: SpoofChecker#RESTRICTION_LEVEL, corresponding to the fifth bit (16). The possible checks are:

• RESTRICTION_LEVEL: flags strings that violate the Restriction Level test as specified in UTS 39; in most cases, this means flagging strings that contain characters from multiple different scripts.
• INVISIBLE: flags strings that contain invisible characters, such as zero-width spaces, or character sequences that are likely not to display, such as multiple occurrences of the same non-spacing mark.
• CHAR_LIMIT: flags strings that contain characters outside of a specified set of acceptable characters. See SpoofChecker.Builder#setAllowedChars and SpoofChecker.Builder#setAllowedLocales.
• MIXED_NUMBERS: flags strings that contain digits from multiple different numbering systems.

These checks can be enabled independently of each other. For example, if you were interested in checking for only the INVISIBLE and MIXED_NUMBERS conditions, you could do:

SpoofChecker sc = new SpoofChecker.Builder()
    .setChecks(SpoofChecker.INVISIBLE | SpoofChecker.MIXED_NUMBERS)
    .build();
boolean result = sc.failsChecks("৪8");
System.out.println(result);  // true

Advanced bidirectional usage

If the paragraph direction with which the identifiers will be displayed is not known, there are multiple options for confusable detection depending on the circumstances.

In some circumstances, the only concern is confusion between identifiers displayed with the same paragraph direction.

An example is the case where identifiers are usernames prefixed with the @ symbol. That symbol will appear to the left in a left-to-right context, and to the right in a right-to-left context, so that an identifier displayed in a left-to-right context can never be confused with an identifier displayed in a right-to-left context:

• The usernames "A1א" (A one aleph) and "Aא1" (A aleph 1) would be considered confusable, since they both appear as @A1א in a left-to-right context, and the usernames "אA_1" (aleph A underscore one) and "א1_A" (aleph one underscore A) would be considered confusable, since they both appear as A_1א@ in a right-to-left context.
• The username "Mark_" would not be considered confusable with the username "_Mark", even though the latter would appear as Mark_@ in a right-to-left context, and the former as @Mark_ in a left-to-right context.

In that case, the caller should check for both LTR-confusability and RTL-confusability:

boolean confusableInEitherDirection =
    sc.areConfusable(Bidi.DIRECTION_LEFT_TO_RIGHT, id1, id2) ||
    sc.areConfusable(Bidi.DIRECTION_RIGHT_TO_LEFT, id1, id2);

If the bidiSkeleton is used, the LTR and RTL skeleta should be kept separately and compared, LTR with LTR and RTL with RTL.

In cases where confusability between the visual appearances of an identifier displayed in a left-to-right context with another identifier displayed in a right-to-left context is a concern, the LTR skeleton of one can be compared with the RTL skeleton of the other. However, this very broad definition of confusability may have unexpected results; for instance, it treats the ASCII identifiers "Mark_" and "_Mark" as confusable.

Additional Information

A SpoofChecker instance may be used repeatedly to perform checks on any number of identifiers.

Thread Safety: The methods on SpoofChecker objects are thread safe. The test functions for checking a single identifier, or for testing whether two identifiers are potentially confusable, may called concurrently from multiple threads using the same SpoofChecker instance.