the concept of batched run might not make sense anymore in Fabricator, since copilot is moving away from request-based billing to token-based.
but many of the other providers still have some concept of subscription billing, and I suspect that we might see subscriptions coming back (since subscriptions are good for business, i.e. negative working capital). once the current financial pressure of inference eases a bit.
so maybe I’ll keep the batching code intact, but stop using it for now in Fabricator until it makes sense.
I’ll also have to start thinking about cleaning up my inputs, and the reasoning level that’s used (higher produces more intermediate tokens, which may not always be necessary).
for now, I don’t think I need to react right away, especially since my AI costs are within my monthly budget. So I can worry about efficiency and optimization later, as necessary, and continue to focus on increasing throughput on my task backlog.