mistokenize
mistokenize verb: to tokenize incorrectly.
Verb
mistokenize (third-person singular simple present mistokenizes, present participle mistokenizing, simple past and past participle mistokenized)
- To tokenize incorrectly.
- C++ considers ::*, .* and ->* each to be a single token and a single operator. Some pre-Release 2.0 implementations mistokenize expressions involving pointer-to-pointer-to-member.1994, Mark A. Terribile, Practical C++, page 109
- These were mostly proper names, such as Ronny Johnsen, or foreign language items such as ambre solaire (French) and fairie queene (Middle English), as well as a few misspelt or mistokenized items.2016, Hans-Jörg Schmid, Entrenchment and the Psychology of Language Learning, page 111
Forms of mistokenize
| third-person singular present | mistokenizes |
|---|---|
| present participle | mistokenizing |
| past participle | mistokenized |
| past | mistokenized |
Origin of mistokenize
From mis- + tokenize.
Questions about mistokenize
- What does mistokenize mean?
- Mistokenize means to tokenize incorrectly.
- What part of speech is mistokenize?
- Mistokenize is a verb.
- What is the past tense of mistokenize?
- The past tense of mistokenize is mistokenized.
Words near mistokenize in the dictionary
Source and license
Adapted from Wiktionary: mistokenize (page history and authors). Text available under the Creative Commons Attribution-ShareAlike 4.0 license. Word.ws reformatted the entry and added links. Wiktionary data of 2026-09-02.
Last updated:
Cite this entry: “mistokenize.” Word.ws, https://word.ws/mistokenize. Based on Wiktionary, CC BY-SA 4.0.