kestrel/kestrel-7b-gguf

Overview

Quantised builds of kestrel-7b-instruct for local runtimes, from 3-bit to 8-bit.

kestrel/kestrel-7b-gguf is a 7B-parameter chat model, published in gguf under the apache-2.0 license. It builds on kestrel/kestrel-7b-instruct.

Intended use

• Chat in en, es, fr.
• Research, prototypes and products that keep a person in the loop.
• Fine-tuning as a starting point for a narrower task.

How to use

# Command-line client (planned; the shape may change)
pace pull kestrel/kestrel-7b-gguf

# Python (planned)
from openpace import load
model = load("kestrel/kestrel-7b-gguf")

Limitations

• It can state wrong things fluently. Check facts that matter.
• Quality drops on languages and domains that were thin in its training data.
• It reflects biases present in public text.

License

Released under apache-2.0. Read the license file before using the weights commercially.