This is an important article. I hadn’t realized it was already getting this bad. Like a frog enjoying a nice warm bath ...
> Most people do not switch their operating system or phone provider every week either. But even if you do not utilize that freedom, it matters because it changes the relationship you have with the provider and the provider has with you.
This is why it’s important to utilize your freedoms. Do NOT let yourself get locked into a particular ecosystem (this is why I’m building a phone app for OpenCode).
This article makes me reconsider using my recently acquired Codex sub in my home setup. I never liked that they hide the reasoning, but somehow overrode the cognitive dissonance because the performance is so good. But the inauditability is already a huge problem.
placebo 2 hours ago [-]
>Like a frog enjoying a nice warm bath...
Exactly :-)
Call me naive but I think dark patterns are a short term strategy for winning and I'm optimistic that in the long run they will be replaced with those that are more respectful and oriented to the greater good (granted, the long run might take more time than one hopes for).
Given that the pendulum can sometime swing back fast enough to be able leverage it, it might be a good time to focus on models that are both open weights and economically viable to run to build the next great thing.
blauditore 30 minutes ago [-]
> [...] dark patterns are a short term strategy for winning and I'm optimistic that in the long run they will be replaced with those that are more respectful and oriented to the greater good
This is definitely not the way most software has been going in the past decades. It's rather the opposite: Companies play friendly to obtain a user base, then start applying more and more dark patters to further increase their profit. Facebook, Evernote, Instagram, Komoot... The list goes on and on.
And yet people still go with Sign in with Google as the only login method.
msdz 2 hours ago [-]
I’ve actually started ignoring services where that’s the only login possibility.
Sorry even if your platform is the greatest thing ever, but I’ll find a different tool. I’ve read one too many stories about Google (or Apple!) closing the entire account over some bullshit unnecessary reason like “fraudulent” gift card issues or whatever.
I’m certain the affected people would’ve preferred to just pay back the amount in question instead of losing their entire Google Drive, or their 20 years of iCloud Photos or whatever.
n6242 46 minutes ago [-]
It's the only reason I want to replace Tailscale with something else or look into self-host when have a bit of time during my vacation. They only allow login through a third party, which is a big no for me.
corney91 35 minutes ago [-]
Tbf to Tailscale they allow any OIDC provider[1]. I wish this was more normalised, then we could have one login everywhere regardless of who hosts it (even if it's yourself).
It’s odd, I just saw my first one of these today. Some former Figma person linked an AI site comp designer.
I was going to give it a shot, but the only choice was a Google login.
So very weird.
wwind123 1 hours ago [-]
I am guessing that's because Sign in with Google is the least of the evil, as compared to other popular OAuth Sign-in options like Facebook, X...
qurren 2 hours ago [-]
Because with the alternative they'll require you to "confirm your e-mail" instead of just picking a password
eru 2 hours ago [-]
I'm actually more annoyed by having to pick passwords than confirming my email.
mystifyingpoi 1 hours ago [-]
I'm opposite. If my email goes down, or I do not have access to it, I cannot login. With passwords I still can.
lucumo 55 minutes ago [-]
Doesn't your password manager take care of that?
hotelsacher 2 hours ago [-]
[dead]
hobofan 2 hours ago [-]
I think the article gives a very good overview of a problem that most users of AI rarely evaluate / have to grapple with.
There really is a surprising amount of coupling that happens with many of the "frontier inference providers", where a lot of the powerful non-LLM extensions (web search, code execution) are packaged as simple "tools" on the surface, that build up a lot of moat. Those are parts that are in theory nicely separable from the inference API, and could be externalized via MCP servers, but are usually not offered as such by the inference providers themselves, and are often only available in a slightly less powerful variant from other providers.
We've faced that issue repeatedly while building a on-premise provider-agnostic Chat UI & platform[0], where even adding something as simple as an in-chat image generation tool for the end-users (which is just a build-in tool in the OpenAI Responses API), becomes a bit of an ordeal (though part of that is due to the MCP spec missing a native file transfer protocol as of today[1]).
I am quite hopeful though, as with recent shifts of interest towards open weight models, there will be more opportunities for companies offering alternative implementations in a easier plug-and-play manner.
I'm in complete agreement that things like encrypted sub-agent messages, etc, are terrible, especially because it is fundamentally irresponsible to have an agent running on your machine without the slightest hint of what the prompt is.
That said, I don't really see a problem with hosted tools being offered by providers. They're like impulse items at checkout.
You shouldn't implement image generation as MCP: just write your own tool. There are plenty of image/media inference providers (e.g. Fal), web search or deep research providers, etc.
hobofan 38 minutes ago [-]
> You shouldn't implement image generation as MCP: just write your own tool. There are plenty of image/media inference providers (e.g. Fal), web search or deep research providers, etc.
Yes, and we as "harness providers" don't want to have to hard-code integrations for each of them, and would rather provide an out-of-the box baseline in the form on an MCP server that our customers can either extend or replace with MCP servers by third-party providers. That helps us focus on the core of our offering, while also giving our customers good flexibility, which so far has been working out great.
That's exactly what standards/protocols like MCP are for. You likely nowadays also wouldn't suggest that someone builds bespoke logging/tracing integrations for each provider on the market while the good generic solution in building OTEL logging/tracing exists.
the_mitsuhiko 44 minutes ago [-]
Hosted tools are absolutely fine. Hiding away their results in encrypted blobs less so.
skybrian 1 hours ago [-]
I don’t see this as a big deal in practice. Conversations contain a bunch of junk anyway, so removing it from the context is usually good.
In my repo, I have a notes directory. I ask the AI to write a markdown file with what it learned, what work has been done, and what remains. In the next conversation, I can ask another model to pick it up from there. Sometimes I edit the note first.
mnewme 3 hours ago [-]
Thank you Armin, Mario and team for the great writeup, absolutely agree and I love what you guys are building
swyx 2 hours ago [-]
what would it take for this to be more than just hn blog fodder?
the_mitsuhiko 1 hours ago [-]
We were thinking about it while writing it. I think one thing at least open source harnesses can do is push back on using some of these conveniences from the providers. For instance the moment we start compromising on the idea of portable sessions, then there will be less pressure (For instance people asked for OpenAI server side compaction or built-in native web_search support).
The real thing that will take this alive though is people pushing back a bit against some of these newfangled APIs. Now that there is real competition from the new generation of Chinese models which have much fewer of those restrictions, I think there might be a moment.
steilpass 37 seconds ago [-]
What are features that only work with session transparency / ownership?
e.g.
- with model XYZ and their open reasoning token we are able to improve our skill ABC by 20%
- cross checking between models improves results by 10% and is only possible with full context visibility
surgical_fire 28 minutes ago [-]
I recently moved entirely to Pi from Claude Code. There's a bit of more upfront work in that I am taking the time to customize it to the way I want it to work, but I
it is clear that it will benefit me in the end. It adds a level of intentionality to everything; all the building blocks I am adding to it in terms of templates, extensions, and skills are, obviously, things I want.
And I have been using Chinese models with it. It was not anything ideological. Those models are both excellent and cheap. Hard to beat that combo.
But on top of that, the sort of output I am getting in Pi in relation to what I got in CC is refreshing, in that nothing seems to be hidden from me.
And yeah, the ability to switch models mid-session is an interesting one, especially with branching; where I can branch to switch to a different model and still go back to the original branch with the original model.
bitpush 2 hours ago [-]
Ouch, seems a bit harsh. They are building something, and with their track record we can be hopeful.
padolsey 4 hours ago [-]
>A user should be able to close an account, keep a session, and hand it to another model. The new model may disagree, ask questions, or perform worse.
I think this is a fair contract. I also think a user should ideally be able to easily identify a comparable model in terms of embedding 'signature'. When GPT-4o originally kicked the bucket, I remember reading lots of anecdotes of people desperately searching for models similar in manner and language, so they could pump in their exports and re-find their friend. Other open-ai models just didn't have the same vibe. It was sad to read. This, to me, is the power of open-weights models. They are for perpetuity. You can keep your guide, your friend, your therapist, whatever. No big company can pull the rug.
bob1029 2 hours ago [-]
OpenAI has been nudging their users toward the Responses API lately. Chat completion can still do everything if you are clever. The obvious reason to push everyone into the Responses API is that the reasoning process can be locked down and encrypted. The semantics of the chat completion API are fundamentally transparent. 100% of the tokens are visible.
If you have patience and the willingness to endure a little bit of pain, you can still retain autonomy over the entire reasoning process while using the latest 5.6 model family. The only downside is that you are now fully responsible for it.
Consider that when you flip your agent's reasoning level to "xhigh" or whatever, it's not some magical model internals being pushed around. There isn't an actual "try harder" knob on the black box. This is merely orchestration of many instances of one or more model types based upon some proprietary harness logic. The chances you can develop a domain specific reasoning process that outperforms the frontier providers is still very good.
dannyw 45 minutes ago [-]
You don't see real reasoning tokens with the Chat Completions API; and 'store: false' with Responses really isn't that different to Completions.
FWIW, if you have some tokens to spend, you might want to test Responses vs Completions in intelligence. Since GPT-5 models, we've consistently seen small, but statistically significant and reproducible improvements in intelligence with Responses API vs Completions.
However it works underneath the hood, it's real.
hobofan 1 hours ago [-]
> Chat completion can still do everything if you are clever.
On a purely functional level, yes. However for interactive use cases, the Completions API, as provided by OpenAI or Azure, if paired with reasoning effort of any kind, provides an awful user experience, as you will have a perceived delay of 10+ seconds until the first tokens stream in.
If using other providers that are exposing their thinking traces, this is less of an issue, as they've just extended the Comletions API format to have delta events with reasoning_content.
charcircuit 1 hours ago [-]
OpenAI has hidden its reasoning for almost 2 years.
gblargg 2 hours ago [-]
BTW the web page uses massive iGPU if you don't have a fast machine.
sdoering 56 minutes ago [-]
> “Most People Do Not Switch Models Mid-Session”
I do, quite regularly. Because different models have different strength (for example when producing text for live presentations based on the text for a reading deck). Or when it comes to other aspects of the work. I regularly switch between open wheight models and closed models.
I know, I am a tiny minority here. And this behavior only ever started a few weeks ago. But it quickly became a habbit, to CTRL-L in pi and change the model.
iwassayinbourns 51 minutes ago [-]
Out of interest, what models do you find work best for producing text for live presentations based on the text for a reading deck?
“A classic pattern in technology economics, identified by Joel Spolsky, is layers of the stack attempting to become monopolies while turning other layers into perfectly-competitive markets which are commoditized, in order to harvest most of the consumer surplus; discussion and examples”
throwaway63467 2 hours ago [-]
I think most LLM work will migrate to open weight models in a couple of years for this exact reason (as well as price), unless the big providers manage to have their way and the market gets regulated in a way that favors closed source models. To me it seems though like most models are becoming good enough for most tasks we want them to do so no need to use the latest super intelligent one that knows how to compose operas and do quantum chemistry and have it answer support requests. And with these models transparency is no issue.
ggm 2 hours ago [-]
Is this a analogous to idempotent behaviour? Of reproducible compilation? If the inputs and outputs aren't sufficient to reproduce one from the other, what is a minimally satisfying description of the middle third, the missing part? The "other" inputs?
wseqyrku 19 minutes ago [-]
> Encryption for Whom?
This one is not particularly limited to LLMs. The entire cloud infrastructure is built around the trust me bro model.
captainmuon 3 hours ago [-]
It's one thing that the APIs don't return the reasoning tokens in a readable fashion. But then why do I have to send them the whole conversation every time? If they don't allow me to see parts of the conversation then they should have the burden of storing and transmitting it, and shouldn't charge more and more the longer the conversation gets (unless opaque token caching kicks in) like some LLM Shlemil the painter.
Encrypted reasoning content is one of the things we keep as blobs in the transcript, even though we can only send it back to the original provider that created them, as otherwise we cannot keep caches live.
captainmuon 59 minutes ago [-]
Right, but my point was, I should be able to just send something like
Seems like it would be even less work for the LLM host to just check if ID 123 is still in a cache (and if not, load it from a database) than decoding my request and checking cryptographic signatures. Right now the whole conversation history with blobs functions like a really long ID.
the_mitsuhiko 32 minutes ago [-]
But then on a cache miss you need to resend the whole thing. Cached websocket connections on OpenAI work like this in a way. But once you lose the connection then you lose that.
Your idea though is the core of the responses API with store where everything is stored on their side and then you append to it.
For the LLM there is no difference btw. In either way you need to locate the cache and use it.
charcircuit 2 hours ago [-]
>and shouldn't charge more and more the longer the conversation gets
The cost of generating token N is O(N) with KV cache so it's unrealistic to expect to not be charged more the longer the conversation is if you are looking for the minimum price.
captainmuon 47 minutes ago [-]
Except, if some of the answered tokens are opaque to me, but I pay for them, the operator has no incentive to keep their number low [1]. For all I know the model could be spinning thumbs while thinking. If they cannot provide transparency, I would prefer that they not charge me for that part. Of course, they would prefer to charge me, so sure it is unrealistic to expect that they don't. I'm not trying to predict what they are going to do as a market participant, just stating my moral preference :-).
Also, isn't it without caching even something like O(N^2) because you have to replay the whole conversation on every request to reach the same internal state? My point was I shouldn't have to pay for cache misses if hitting the cache is not deterministic. Give me a guarantee that you keep the session "hot" for N minutes, and cached on disk for M months, charge a little bit more on average, but then the pricing is at least transparent.
[1] Except the general market pressure to keep total cost for the same problem solved lower than the competition.
n0on3 3 hours ago [-]
I get the point of the article and it resonates with my general tendency to support more openness, but older and more cynical me wonders how much of this is a battle worth fighting or a self-serving argument from an harness developer.
I mean, the article says each of the points it is complaining about has “_a basic justification that's trivial for a provider to come up with, along with good arguments for why this is good for the user_“, which to me sounds like implying these are just bs to have people accept them, but is that actually the case?
For instance, this [0] was mentioned here a while ago, and based on that it seems pretty clear why one would choose not to provide the _thinking_ anymore…
> these are just bs to have people accept them, but is that actually the case?
I tried to outline that in my sibling comment[0], but I think the article also gives a good example in the "hidden searches" paragraph: There is really no good reason not to expose details about many of the builtin tools, or provide them as standalone services, other than protect their moat.
I don't think that "role confusion" attacks could be meaningful prevented by not showing reasoning traces. Prevention of reasoning traces is about as futile as prevention of system prompt leakage. And here you would just have to manage to exfiltrate a small number of reasoning traces per model family in order to distill the writing style of thinking traces.
you can port your phone number to any carrier but your AI doesn't know your name until you explain it from scratch
jokiruiz 1 hours ago [-]
[flagged]
alvink1212 56 minutes ago [-]
[flagged]
yucongchen 1 hours ago [-]
[flagged]
thegeomaster 2 hours ago [-]
If even Earendil is publishing AI generated blog posts...
the_mitsuhiko 2 hours ago [-]
We do not.
buttavia99 34 minutes ago [-]
according to both gptzero and pangram (and anyone that ever read AI-generated text), you do.
dyates 2 hours ago [-]
We seem to be reaching a point where the use of formatting and the odd rhetorical flourish gets you accused of AI-generated writing. Sad.
try-working 56 minutes ago [-]
OpenAI is doing this so that you can't preserve cache when routing between for example GPT 5.6 and DeepSeek V4, or other providers. I'm building a router that easily reaches >99% cache hit rate in Pi when routing between GPT 5.4, Kimi, DeepSeek etc. Working on a big update that's coming out in a week or so.
> Most people do not switch their operating system or phone provider every week either. But even if you do not utilize that freedom, it matters because it changes the relationship you have with the provider and the provider has with you.
This is why it’s important to utilize your freedoms. Do NOT let yourself get locked into a particular ecosystem (this is why I’m building a phone app for OpenCode).
This article makes me reconsider using my recently acquired Codex sub in my home setup. I never liked that they hide the reasoning, but somehow overrode the cognitive dissonance because the performance is so good. But the inauditability is already a huge problem.
Exactly :-)
Call me naive but I think dark patterns are a short term strategy for winning and I'm optimistic that in the long run they will be replaced with those that are more respectful and oriented to the greater good (granted, the long run might take more time than one hopes for).
Given that the pendulum can sometime swing back fast enough to be able leverage it, it might be a good time to focus on models that are both open weights and economically viable to run to build the next great thing.
This is definitely not the way most software has been going in the past decades. It's rather the opposite: Companies play friendly to obtain a user base, then start applying more and more dark patters to further increase their profit. Facebook, Evernote, Instagram, Komoot... The list goes on and on.
This is the way.
Sorry even if your platform is the greatest thing ever, but I’ll find a different tool. I’ve read one too many stories about Google (or Apple!) closing the entire account over some bullshit unnecessary reason like “fraudulent” gift card issues or whatever. I’m certain the affected people would’ve preferred to just pay back the amount in question instead of losing their entire Google Drive, or their 20 years of iCloud Photos or whatever.
[1] https://tailscale.com/docs/integrations/identity/custom-oidc
I was going to give it a shot, but the only choice was a Google login.
So very weird.
There really is a surprising amount of coupling that happens with many of the "frontier inference providers", where a lot of the powerful non-LLM extensions (web search, code execution) are packaged as simple "tools" on the surface, that build up a lot of moat. Those are parts that are in theory nicely separable from the inference API, and could be externalized via MCP servers, but are usually not offered as such by the inference providers themselves, and are often only available in a slightly less powerful variant from other providers.
We've faced that issue repeatedly while building a on-premise provider-agnostic Chat UI & platform[0], where even adding something as simple as an in-chat image generation tool for the end-users (which is just a build-in tool in the OpenAI Responses API), becomes a bit of an ordeal (though part of that is due to the MCP spec missing a native file transfer protocol as of today[1]).
I am quite hopeful though, as with recent shifts of interest towards open weight models, there will be more opportunities for companies offering alternative implementations in a easier plug-and-play manner.
[0]: https://github.com/EratoLab/erato
[1]: https://github.com/modelcontextprotocol/modelcontextprotocol...
That said, I don't really see a problem with hosted tools being offered by providers. They're like impulse items at checkout.
You shouldn't implement image generation as MCP: just write your own tool. There are plenty of image/media inference providers (e.g. Fal), web search or deep research providers, etc.
Yes, and we as "harness providers" don't want to have to hard-code integrations for each of them, and would rather provide an out-of-the box baseline in the form on an MCP server that our customers can either extend or replace with MCP servers by third-party providers. That helps us focus on the core of our offering, while also giving our customers good flexibility, which so far has been working out great.
That's exactly what standards/protocols like MCP are for. You likely nowadays also wouldn't suggest that someone builds bespoke logging/tracing integrations for each provider on the market while the good generic solution in building OTEL logging/tracing exists.
In my repo, I have a notes directory. I ask the AI to write a markdown file with what it learned, what work has been done, and what remains. In the next conversation, I can ask another model to pick it up from there. Sometimes I edit the note first.
The real thing that will take this alive though is people pushing back a bit against some of these newfangled APIs. Now that there is real competition from the new generation of Chinese models which have much fewer of those restrictions, I think there might be a moment.
e.g. - with model XYZ and their open reasoning token we are able to improve our skill ABC by 20% - cross checking between models improves results by 10% and is only possible with full context visibility
And I have been using Chinese models with it. It was not anything ideological. Those models are both excellent and cheap. Hard to beat that combo.
But on top of that, the sort of output I am getting in Pi in relation to what I got in CC is refreshing, in that nothing seems to be hidden from me.
And yeah, the ability to switch models mid-session is an interesting one, especially with branching; where I can branch to switch to a different model and still go back to the original branch with the original model.
I think this is a fair contract. I also think a user should ideally be able to easily identify a comparable model in terms of embedding 'signature'. When GPT-4o originally kicked the bucket, I remember reading lots of anecdotes of people desperately searching for models similar in manner and language, so they could pump in their exports and re-find their friend. Other open-ai models just didn't have the same vibe. It was sad to read. This, to me, is the power of open-weights models. They are for perpetuity. You can keep your guide, your friend, your therapist, whatever. No big company can pull the rug.
If you have patience and the willingness to endure a little bit of pain, you can still retain autonomy over the entire reasoning process while using the latest 5.6 model family. The only downside is that you are now fully responsible for it.
Consider that when you flip your agent's reasoning level to "xhigh" or whatever, it's not some magical model internals being pushed around. There isn't an actual "try harder" knob on the black box. This is merely orchestration of many instances of one or more model types based upon some proprietary harness logic. The chances you can develop a domain specific reasoning process that outperforms the frontier providers is still very good.
FWIW, if you have some tokens to spend, you might want to test Responses vs Completions in intelligence. Since GPT-5 models, we've consistently seen small, but statistically significant and reproducible improvements in intelligence with Responses API vs Completions.
However it works underneath the hood, it's real.
On a purely functional level, yes. However for interactive use cases, the Completions API, as provided by OpenAI or Azure, if paired with reasoning effort of any kind, provides an awful user experience, as you will have a perceived delay of 10+ seconds until the first tokens stream in.
If using other providers that are exposing their thinking traces, this is less of an issue, as they've just extended the Comletions API format to have delta events with reasoning_content.
I do, quite regularly. Because different models have different strength (for example when producing text for live presentations based on the text for a reading deck). Or when it comes to other aspects of the work. I regularly switch between open wheight models and closed models.
I know, I am a tiny minority here. And this behavior only ever started a few weeks ago. But it quickly became a habbit, to CTRL-L in pi and change the model.
This one is not particularly limited to LLMs. The entire cloud infrastructure is built around the trust me bro model.
Encrypted reasoning content is one of the things we keep as blobs in the transcript, even though we can only send it back to the original provider that created them, as otherwise we cannot keep caches live.
Your idea though is the core of the responses API with store where everything is stored on their side and then you append to it.
For the LLM there is no difference btw. In either way you need to locate the cache and use it.
The cost of generating token N is O(N) with KV cache so it's unrealistic to expect to not be charged more the longer the conversation is if you are looking for the minimum price.
Also, isn't it without caching even something like O(N^2) because you have to replay the whole conversation on every request to reach the same internal state? My point was I shouldn't have to pay for cache misses if hitting the cache is not deterministic. Give me a guarantee that you keep the session "hot" for N minutes, and cached on disk for M months, charge a little bit more on average, but then the pricing is at least transparent.
[1] Except the general market pressure to keep total cost for the same problem solved lower than the competition.
I mean, the article says each of the points it is complaining about has “_a basic justification that's trivial for a provider to come up with, along with good arguments for why this is good for the user_“, which to me sounds like implying these are just bs to have people accept them, but is that actually the case?
For instance, this [0] was mentioned here a while ago, and based on that it seems pretty clear why one would choose not to provide the _thinking_ anymore…
[0] https://role-confusion.github.io/
I tried to outline that in my sibling comment[0], but I think the article also gives a good example in the "hidden searches" paragraph: There is really no good reason not to expose details about many of the builtin tools, or provide them as standalone services, other than protect their moat.
I don't think that "role confusion" attacks could be meaningful prevented by not showing reasoning traces. Prevention of reasoning traces is about as futile as prevention of system prompt leakage. And here you would just have to manage to exfiltrate a small number of reasoning traces per model family in order to distill the writing style of thinking traces.
[0]: https://news.ycombinator.com/item?id=49120015