Bobbie-model _verified_ Info

| Benchmark | Bobbie-7B | Llama-3-8B | Mistral-7B | |-----------|-----------|------------|------------| | MMLU (5-shot) | 64.2 | 66.7 | 63.9 | | GSM8K (8-shot) | 52.8 | 54.9 | 50.3 | | HumanEval (pass@1) | 32.5 | 34.2 | 31.8 | | | 82.3 | 67.1 | 71.4 | | Inference tokens/sec | 98 | 72 | 88 |

Bobbie is not just another incremental fine-tune. It represents a thoughtful experiment in . bobbie-model

messages = [ "role": "user", "content": "Summarize this 20k token document..." ] inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device) output = model.generate(inputs, max_new_tokens=512, temperature=0.7) print(tokenizer.decode(output[0][inputs.shape[1]:])) Bobbie works out-of-the-box with vLLM 0.6.0+: | Benchmark | Bobbie-7B | Llama-3-8B | Mistral-7B

If you’ve been following the open-source LLM space, you’ve likely memorized the specs of Llama 3, Mixtral, and Qwen. But a new contender has been quietly gaining traction in the "small model" category: . But a new contender has been quietly gaining