GPT-3.5 crashes when it thinks about useRalativeImagePath too much (iter.ca)

Article summary

GPT-3.5 crashes when it encounters the token 'useRalativeImagePath', which is likely due to the token being present in the dataset used to generate the tokenizer but not in the training data. This results in the model not understanding the token's semantics. The issue does not occur with GPT-4, which may use a different tokenizer or training data. The token's presence in XML configuration files for Katalon Studio and its misspelling may have contributed to its inclusion as a separate token.

Main themes

• GPT-3.5 crash
• tokenizer behavior
• glitch tokens
• AI safety
• language model limitations
• adversarial attacks

What commenters say

• The crash may be due to the model's inability to handle rare tokens, which can cause it to output invalid or nonsensical responses.
• The issue could be related to the tokenizer's behavior, particularly with regard to special tokens or blacklisted words.
• The presence of 'glitch tokens' like 'useRalativeImagePath' can be used to test the model's limits and potentially identify vulnerabilities.
• The idea of intentionally introducing 'kill words' or 'kill images' into AI models as a safety measure is proposed, but its feasibility and potential consequences are debated.
• The discussion highlights the importance of understanding the tokenizer and training data used in language models, as well as the potential risks and limitations of relying on these models.
• Some commenters speculate that the token may have been blacklisted after a previous incident, which could be the cause of the crash.
• The possibility of using 'glitch tokens' to test or manipulate AI models raises concerns about the potential for adversarial attacks or data poisoning.