The session you cannot take with you (earendil.com)

Article summary

The article discusses the trend of inference APIs moving away from providing portable sessions, where a user can take their conversation history and continue it with a different model or provider. This shift is driven by features such as encrypted reasoning tokens, hosted searches, and opaque compaction, which prioritize provider convenience over user control. As a result, users are locked into a particular ecosystem, losing the ability to inspect, export, or replay their sessions. The authors argue that this direction has significant implications for user freedom and provider accountability.

Main themes

• Portable sessions
• Inference API design
• User control and freedom
• Provider lock-in
• Encryption and transparency
• Single-sign-on and authentication

What commenters say

• The lack of portable sessions is a significant issue that affects user freedom and provider accountability.
• Some features that limit portability, such as encrypted reasoning tokens, may have legitimate justifications but also serve to lock users into a particular ecosystem.
• The benefits of portable sessions, such as improved analytics and model switching, are not being prioritized by providers.
• Users should be cautious of services that only offer single-sign-on options, as this can lead to lock-in and loss of control over their data.
• Password managers can mitigate some of the issues with traditional login methods, but may not be foolproof.
• Some users prioritize the convenience of single-sign-on options over the potential risks of lock-in and data loss.
• The trend towards non-portable sessions may be driven by providers' desire to harvest consumer surplus and maintain a competitive advantage.
• Open-source harnesses and alternative models can help push back against the trend of non-portable sessions and promote greater transparency and user control.