Measuring Claude 4.7's tokenizer costs (claudecodecamp.com)

Article summary

The article measures the cost of Anthropic's new tokenizer in Claude 4.7, finding it uses 1.47x more tokens than the previous version, which can lead to increased costs for users. The author also tests whether the new tokenizer improves instruction following, finding a small but real improvement. The increased token usage can lead to higher costs, especially for users who hit rate limits. The article concludes that the increased cost may be worth it for some users, depending on their specific use case.

Main themes

• Tokenizer costs
• Instruction following
• Model updates
• Cost-benefit analysis
• LLM performance

What commenters say

• The new tokenizer's increased cost may be justified by its improved performance, but it depends on the specific use case.
• Some users may not notice a significant difference in performance between the old and new models, and may prefer to stick with the older version to save on costs.
• The increased cost of the new tokenizer may be a sign of diminishing returns on investment in LLMs, with higher performance coming at a higher cost.
• The decision to upgrade to the new model should be based on a careful evaluation of the costs and benefits, rather than just following the latest release.
• The performance of LLMs can be difficult to measure and compare, and benchmarks may not always reflect real-world usage.
• The commercialization of LLMs may lead to a focus on profit over ethics and social responsibility, particularly after an IPO.
• Some users may be able to achieve their goals with older models, and may not need to upgrade to the latest version.
• The increased verbosity of the new model's output may be a drawback for some users, who prefer more concise and efficient code.