As AI adoption accelerates, many companies have cut headcount to fund soaring AI token costs, but the expected returns have often failed to materialise. Nvidia CEO Jensen Huang highlighted this tension, noting that an engineer’s annual AI token consumption should not exceed half their salary. Yet, many organisations treat the token budget as fixed while viewing workforce costs as flexible, leading to layoffs that erode institutional knowledge.
Engineering the Token Budget Down
The key to sustainable AI investment lies in engineering the token budget rather than cutting people. Techniques like prompt caching, which avoids reprocessing identical text inputs, can reduce token costs by up to 90%. For instance, security firm ProjectDiscovery boosted its cache hit rate from 7% to 84%, slashing its language model expenses by over half while serving billions of tokens.
Other cost-saving measures include routing tasks to smaller, less expensive AI models when possible, using batch processing discounts, and employing retrieval-augmented generation that sends only relevant data slices to the model. Open-weight models also offer lower-cost alternatives for routine workloads if companies can manage the infrastructure.
Investing Savings in People
Optimising token spend only delivers value if the savings are reinvested productively, especially in people. Research shows organisations that use AI to amplify their workforce, rather than replace it, achieve better returns. Klarna’s experiment replacing 700 customer service roles with AI assistants led to lower customer satisfaction, prompting a blended approach where AI handles routine queries and humans manage complex cases.
Moreover, workforce cuts risk eliminating entry-level roles crucial for training future senior engineers needed to oversee AI systems. Companies that engineer a 60% reduction in token costs create budget room to maintain or expand their teams, a leadership choice rather than a financial constraint.
Ultimately, the most successful companies will be those recognising the token budget as the flexible line. By squeezing token costs through smart engineering and investing the difference in talent, they maximise the value of both their AI spend and their people.
Disclaimer: This article provides strategic insights on AI cost management and workforce investment. It does not offer medical, legal, or financial advice. Organisations should evaluate their specific context before applying these recommendations.
For practical AI adoption and workflow optimisation, explore JASON AI’s solutions at https://jasonjuul.com.