markdown_it.common.utils module

Utilities for parsing source text

markdown_it.common.utils.MD_TRIM_CHARS = '\t\n\x0b\x0c\r \xa0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200a\u2028\u2029\u202f\u205f\u3000'

The characters String.prototype.trim removes in JavaScript, which is what upstream markdown-it strips, minus U+FEFF.

str.strip() without an argument uses str.isspace(), which also removes U+001C, U+001D, U+001E, U+001F and U+0085. Those are not whitespace in CommonMark and are not removed by trim, so relying on it drops them from the output and makes two different reference labels compare equal.

U+FEFF is deliberately excluded: trim does remove it, and that has the very label folding effect this constant exists to avoid.

markdown_it.common.utils.MD_TRIM_RE = re.compile('[\\\t\\\n\\\x0b\\\x0c\\\r\\ \xa0\u1680\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200a\u2028\u2029\u202f\u205f\u3000]+')

A run of one or more characters from MD_TRIM_CHARS.

markdown_it.common.utils.arrayReplaceAt(src: list[_ItemTV], pos: int, newElements: list[_ItemTV]) → list[_ItemTV]

Remove element from array and put another array at those position. Useful for some operations with tokens

markdown_it.common.utils.charCodeAt(src: str, pos: int) → int | None

Returns the Unicode value of the character at the specified location.

@param - index The zero-based index of the desired character. If there is no character at the specified index, NaN is returned.

This was added for compatibility with python

markdown_it.common.utils.charStrAt(src: str, pos: int) → str | None

Returns the Unicode value of the character at the specified location.

@param - index The zero-based index of the desired character. If there is no character at the specified index, NaN is returned.

This was added for compatibility with python

markdown_it.common.utils.escapeHtml(raw: str) → str

Replace special characters “&”, “<”, “>” and ‘”’ to HTML-safe sequences.

markdown_it.common.utils.escapeRE(string: str) → str

markdown_it.common.utils.fromCodePoint(c: int) → str

Convert ordinal to unicode.

Note, in the original Javascript two string characters were required, for codepoints larger than 0xFFFF. But Python 3 can represent any unicode codepoint in one character.

markdown_it.common.utils.isLinkClose(string: str) → bool

markdown_it.common.utils.isLinkOpen(string: str) → bool

markdown_it.common.utils.isMdAsciiPunct(ch: int) → bool

Markdown ASCII punctuation characters.

!, ", #, $, %, &, ', (, ), *, +, ,, -, ., /, :, ;, <, =, >, ?, @, [, \, ], ^, _, `, {, |, }, or ~

See http://spec.commonmark.org/0.15/#ascii-punctuation-character

Don’t confuse with unicode punctuation !!! It lacks some chars in ascii range.

markdown_it.common.utils.isPunctChar(ch: str) → bool

Check if character is a punctuation character.

markdown_it.common.utils.isSpace(code: int | None) → bool

Check if character code is a whitespace.

markdown_it.common.utils.isStrSpace(ch: str | None) → bool

Check if character is a whitespace.

markdown_it.common.utils.isValidEntityCode(c: int) → bool

markdown_it.common.utils.isWhiteSpace(code: int) → bool

Zs (unicode class) || [tfvrn]

markdown_it.common.utils.mdTrim(string: str) → str

Strip leading and trailing whitespace, using the CommonMark set.

markdown_it.common.utils.normalizeReference(string: str) → str

Helper to unify [reference labels].

markdown_it.common.utils.replaceEntityPattern(match: str, name: str) → str

Convert HTML entity patterns, see https://spec.commonmark.org/0.30/#entity-references

markdown_it.common.utils.stripEscape(string: str) → str

Strip escape characters

markdown_it.common.utils.unescapeAll(string: str) → str