Word.ws English Dictionary

tokeniser

tokeniser is the alternative spelling of tokenizer.

Meaning of tokenizer

Noun

tokenizer (plural tokenizers)

  1. A system that parses an input stream into its component tokens.

Forms of tokeniser

pluraltokenisers

Origin of tokeniser

From tokenise + -er.

Questions about tokeniser

What is tokeniser the alternative spelling of?
Tokeniser is the alternative spelling of tokenizer.

Words near tokeniser in the dictionary

  1. tokenings
  2. tokenisation
  3. tokenisations
  4. tokenise
  5. tokenised
  6. tokeniser
  7. tokenisers
  8. tokenises
  9. tokenish
  10. tokenising
  11. tokenism

Source and license

Adapted from Wiktionary: tokeniser (page history and authors); tokenizer (page history and authors). Text available under the Creative Commons Attribution-ShareAlike 4.0 license. Word.ws reformatted the entry and added links. Wiktionary data of 2026-09-02.

Last updated:

Cite this entry: “tokeniser.” Word.ws, https://word.ws/tokeniser. Based on Wiktionary, CC BY-SA 4.0.