A simple key is a key that is not denoted by the '?' indicator.
Example of simple keys:
  ---
  block simple key: value
  ? not a simple key:
  : { flow simple key: value }
We emit the KEY token before all keys, so when we find a potential
simple key, we try to locate the corresponding ':' indicator.
Simple keys should be limited to a single line and 1024 characters.

Can a simple key start at the current position? A simple key may
start:
- at the beginning of the line, not counting indentation spaces
      (in block context),
- after '{', '[', ',' (in the flow context),
- after '?', ':', '-' (in the block context).
In the block context, this flag also signifies if a block collection
may start at the current position.

A regular expression matching characters which are not in the hexadecimal set (0-9, A-F, a-f).

Check if we need to increase indentation.

Returns true if the next thing on the reader is a block token.

Returns true if the next thing on the reader is a directive, given that the leading '%' has already been checked.

See Also

3.2.3.4. Directives

Returns true if the next thing on the reader is a document-end ("..."). A document-end is always followed immediately by a new line.

Returns true if the next thing on the reader is a document-start ("---"). A document-start is always followed immediately by a new line.

Returns true if the next thing on the reader is a key token.

Returns true if the next thing on the reader is a plain token.

Returns true if the next thing on the reader is a value token.

This is implemented in CharConstants in SnakeYAML Engine

Fetch an alias, which is a reference to an anchor. Aliases take the format:

*(anchor name)

See Also

3.2.2.2. Anchors and Aliases

Fetch an anchor. Anchors take the form:

&(anchor name)

See Also

3.2.2.2. Anchors and Aliases

Fetch an entry in the block style.

See Also

3.2.3.1. Node Styles

Fetch a block scalar (literal or folded).

See Also

3.2.3.1. Node Styles

Fetch a YAML directive. Directives are presentation details that are interpreted as instructions to the processor. YAML defines two kinds of directives, YAML and TAG; all other types are reserved for future use.

See Also

3.2.3.4. Directives

Fetch a document-end token ("...").

Fetch a document indicator, either "---" for "document-start", or else "..." for "document-end. The type is chosen by the given boolean.

Fetch a document-start token ("---").

Fetch a double-quoted (") scalar.

Fetch a flow-style collection end, which is either a sequence or a mapping. The type is determined by the given boolean. A flow-style collection is in a format similar to JSON. Sequences are started by '[' and ended by ']'; mappings are started by '{' and ended by '}'.

See Also

3.2.3.1. Node Styles

Fetch a flow-style collection start, which is either a sequence or a mapping. The type is determined by the given boolean. A flow-style collection is in a format similar to JSON. Sequences are started by '[' and ended by ']'; mappings are started by '{' and ended by '}'.

See Also

3.2.3.1. Node Styles

Fetch an entry in the flow style. Flow-style entries occur either immediately after the start of a collection, or else after a comma.

See Also

3.2.3.1. Node Styles

Fetch a flow scalar (single- or double-quoted).

See Also

3.2.3.1. Node Styles

Fetch a folded scalar, denoted with a greater-than sign. This is the type best used for long content, such as the text of a chapter or description.

See Also

3.2.3.1. Node Styles

Fetch a key in a block-style mapping.

See Also

3.2.3.1. Node Styles

Fetch a literal scalar, denoted with a vertical-bar. This is the type best used for source code and other content, such as binary data, which must be included verbatim.

See Also

3.2.3.1. Node Styles

Fetch one or more tokens from the StreamReader.

Fetch a plain scalar.

Fetch a single-quoted (') scalar.

We always add STREAM-START as the first token and STREAM-END as the last token.

Fetch a tag. Tags take a complex form.

See Also

3.2.1.2. Tags

Fetch a value in a block-style mapping.

See Also

3.2.3.1. Node Styles

Returns true if more tokens should be scanned.

Return the number of the nearest possible simple key. Actually we don't need to loop through the whole dictionary.

Remove the saved possible key position at the current flow level.

The next token may start a simple key. We check if it's possible and save its position. This function is called for ALIAS, ANCHOR, TAG, SCALAR(flow), '[', and '{'.

The YAML 1.1 specification does not restrict characters for anchors and
aliases. This may lead to problems.
see https://bitbucket.org/snakeyaml/snakeyaml/issues/485/alias-names-are-too-permissive-compared-to
This implementation tries to follow https://github.com/yaml/yaml-spec/blob/master/rfc/RFC-0003.md

Scan to the end of the line after a block scalar has been scanned; the only things that are permitted at this time are comments and spaces.

Scans for the indentation of a block scalar implicitly. This mechanism is used only if the block did not explicitly state an indentation to be used.

See Also

9.2.2. Block Indentation Indicator

Scan a block scalar indicator. The block scalar indicator includes two optional components, which may appear in either order. A block indentation indicator is a non-zero digit describing the indentation level of the block scalar to follow. This indentation is an additional number of spaces relative to the current indentation level. A block chomping indicator is a + or -, selecting the chomping mode away from the default (clip) to either -(strip) or +(keep).

See Also

5.3. Indicator Characters, 9.2.2. Block Indentation Indicator, 9.2.3. Block Chomping Indicator

Annotations

@SuppressWarnings:unchecked, rawtypes

Scan a directive name. Directive names are a series of non-space characters.

See Also

7.1. Directives

Scan a flow-style scalar. Flow scalars are presented in one of two forms; first, a flow scalar may be a double-quoted string; second, a flow scalar may be a single-quoted string. style/syntax

See the specification for details.
Note that we loose indentation rules for quoted scalars. Quoted
scalars don't need to adhere indentation because " and ' clearly
mark the beginning and the end of them. Therefore we are less
restrictive then the specification requires. We only need to check
that document separators are not included in scalars.
     

See Also

9.1. Flow Scalar Styles

Scan some number of flow-scalar non-space characters.

Scan a line break, transforming:

'\r\n' : '\n'
'\r' : '\n'
'\n' : '\n'
'\x85' : '\n'
default : ''

Scan a plain scalar.

See the specification for details.
We add an additional restriction for the flow context:
  plain scalars in the flow context cannot contain ',', ':' and '?'.
We also keep track of the `allow_simple_key` flag here.
Indentation rules are loosed for the flow context.

See the specification for details. SnakeYAML and libyaml allow tabs inside plain scalar

Scan a Tag property. A Tag property may be specified in one of three ways: c-verbatim-tag, c-ns-shorthand-tag, or c-ns-non-specific-tag

c-verbatim-tag takes the form !<ns-uri-char+> and must be delivered verbatim (as-is) to the application. In particular, verbatim tags are not subject to tag resolution.

c-ns-shorthand-tag is a valid tag handle followed by a non-empty suffix. If the tag handle is a c-primary-tag-handle ('!') then the suffix must have all exclamation marks properly URI-escaped (%21); otherwise, the string will look like a named tag handle: !foo!bar would be interpreted as (handle="!foo!", suffix="bar").

c-ns-non-specific-tag is always a lone '!'; this is only useful for plain scalars, where its specification means that the scalar MUST be resolved to have type tag:yaml.org,2002:str.

See Also

8.2. Node Tags TODO Note that this method does not enforce rules about local versus global tags!

Scan a %TAG directive's handle. This is YAML's c-tag-handle.

Parameters

startMark:Mark

- beginning of the handle

Returns:String

scanned handle

See Also

7.1.2.2. Tag Handles

Scan a %TAG directive's prefix. This is YAML's ns-tag-prefix.

See Also

Read a %TAG directive value:

s-ignored-space+ c-tag-handle s-ignored-space+ ns-tag-prefix s-l-comments

See Also

7.1.2. “TAG” Directive

Scan a Tag handle. A Tag handle takes one of three forms:

"!" (c-primary-tag-handle)
"!!" (ns-secondary-tag-handle)
"!(name)!" (c-named-tag-handle)

See the specification for details.
For some strange reasons, the specification does not allow '_' in
tag handles. I have allowed it anyway.
     

See Also

,

Scan a Tag URI. This scanning is valid for both local and global tag directives, because both appear to be valid URIs as far as scanning is concerned. The difference may be distinguished later, in parsing. This method will scan for ns-uri-char*, which covers both cases.

This method performs no verification that the scanned URI conforms to any particular kind of URI specification.

See Also

We ignore spaces, line breaks and comments.
If we find a line break in the block context, we set the flag
`allow_simple_key` on.
The byte order mark is stripped if it's the first character in the
stream. We do not yet support BOM inside the stream as the
specification requires. Any such mark will be considered as a part
of the document.
TODO: We need to make tab handling rules more sane. A good rule is
  Tabs cannot precede tokens
  BLOCK-SEQUENCE-START, BLOCK-MAPPING-START, BLOCK-END,
  KEY(block), VALUE(block), BLOCK-ENTRY
So the checking code is
  if <TAB>:
      self.allow_simple_keys = False
We also need to add the check for `allow_simple_keys == True` to
`unwind_indent` before issuing BLOCK-END.
Scanners for block, flow, and plain scalars need to be modified.

Scan a sequence of %-escaped URI escape codes and convert them into a String representing the unescaped values.

See Also

section 2.4, Escaped Encoding

Read a %YAML directive number: this is either the major or the minor part. Stop reading at a non-digit character (usually either '.' or '\n').

See Also

7.1.1. “YAML” Directive,

Remove entries that are no longer possible simple keys. According to
the YAML specification, simple keys
- should be limited to a single line,
- should be no longer than 1024 characters.
Disabling this procedure will allow simple keys of any length and
height (may cause problems if indentation is broken though).

* Handle implicitly ending multiple levels of block nodes by decreased indentation. This function becomes important on lines 4 and 7 of this example:

1) book one:
2)   part one:
3)     chapter one
4)   part two:
5)     chapter one
6)     chapter two
7) book two: