What’s going on with Jev?

Jev has taken the AI world by storm in the last few weeks, so much so, that OpenAI and many others have rushed into providing a Jev like offering that follows that exact same API in most cases. What’s going on?

The airplane view is that Jev filled a notable gap, that of a fast/cheap but also general enough model to be suitable for a large enough set of AI use cases out of the box. Let’s decouple this. Up until now you had two choices when developing AI:

• a general model that works out of the box but is slow and costly
• a small specialised model that works only after specific training and needs support but is fast and cheap

Most organisations and users of course start with the general model first and optimise speed and performance using a different model family i.e. Haiku vs Opus or Luna vs Sol etc, depending on the task. Some tasks like generating a natural response usually requires a more capable model while others like deciding whether the user confirmed its identity a less capable one. However those less capable models were still quite slow and expensive, compared to the small and specialised ones. At the same time, the latter did not work out of the box which meant added costs from the get go in time or resources in order to train them and collect the required data.

What was missing was a fast and general enough model for those exact use cases that did not need the latest frontier model. Jev overcomes the self imposed barrier of speed and cost that Haiku / Luna models have by reducing its generality to only cases that have to do with decisions or in AI parlance, classifications while at the same time being more general than small specialised models. This explains the popularity and surge over the last weeks purely from an economics perspective. Here is a model that performs on par or better out of the box than Luna or Haiku equivalents in exactly the cases where those are mostly used while being faster and cheaper.

You might be wondering at this point, how did Jev pull this through? We do not know the exact details but we make some well informed assumptions to what might be happening.

Let’s start from speed. Restricting a model to only decision outputs, which are one or few tokens each with reasoning disabled, explains most of the speedup. In fact Haiku and Luna at those settings already run in more comparable speeds. The rest of the speed difference has to do with asking multiple questions on the same context, which may or may not happen in practice, but from a speedup perspective it helps show a more favourable picture because you load the context once and answer multiple times. OpenAI addressed that last part with their decisions API.

Then comes out of the box performance. Is Jev a completely different model or architecture, is it some sort of breakthrough? Well, we do not know, but it’s looking quite unlikely. Instead it’s very likely the same architecture as all other LLMs i.e. a decoder model, bigger than most small specialised models and smaller than the entry point options from frontier labs. This is the sweet spot of general enough. It also means that efforts to copy Jev using a smaller model have a ceiling in performance i.e. not general enough model and efforts from frontier labs have a ceiling in speedup as their general models can only be so small to be general.

The last ingredient of Jev is data. We know this is synthetic so to a certain extent, we can consider part of Jev performance somewhat distilled but that may soon not be the case and it’s not the main point. Data wise, Jev has the advantage to only need decision like data so not the entire Internet and beyond. This means it can achieve equal or better performance in decisions as the frontier labs but with one or two orders of magnitude less data. However, they very likely have a polished enough synthetic dataset to give them some defensibility against competition which has not managed to match their performance even though the ingredients seem quite simple.

Let’s conclude, what does this mean for organisations and users alike?

• Jev and decisions models are important in many use cases and can lower your costs and latency for the same performance, so definitely something to pay attention and experiment
• Jev is not a breakthrough new architecture and as such you are not missing the ship if you don’t adapt quickly. Furthermore you can emulate Jev right now with the cheap offering of most providers and no reasoning dial.
• Jev is quite likely to maintain its lead over the competition due to 1) its synthetic data generation recipe now and data in the near future and 2) its general enough offering which protects it somewhat from the general frontier lab offering