Token Tools Documentation

Welcome to Token Tools’s Documentation! Token Tools is a library that can be used by Salve Organization projects to work with Token’s. To get started with Token Tools, visit the Installation page!

Note

This project is under active development and the API will reach its stable form at v1.0.0

Contents:

• Installation
• Variables
  • Token
  • GENERIC_TOKENS
• Functions
  • normal_text_range(full_text: str, text_range: tuple[int, int] = (1, -1)) -> tuple[list[str], tuple[int, int]]
  • only_tokens_in_text_range(tokens: list[Token], text_range: tuple[int, int]) -> list[Token]
  • merge_tokens(tokens: list[Token]) -> list[Token]
  • overwrite_tokens(old_tokens: list[Token], new_tokens: list[Token]) -> list[Token]
  • overwrite_and_merge_tokens(old_tokens: list[Token], new_tokens: list[Token]) -> list[Token]