AQ-5B

AQ-5B is Zyora Labs' own foundation model — built from scratch, not fine-tuned from someone else's weights. It is a decoder-only Transformer designed to understand every educational stream, from Physics and Maths to Botany, Zoology, Tamil, Engineering and Arts & Science.

Rather than memorising a specific syllabus, AQ-5B is trained on broad conceptual knowledge — base to advanced, so it forms a durable foundation for the next generation of ZyoraByte and Zyora AI products.

AQ-5B is a decoder-only Transformer — RoPE + RMSNorm + SwiGLU + GQA — implemented in pure PyTorch, hand-coded in torch.nn rather than pulled from HuggingFace transformers. The config is kept HF-compatible so the model can be loaded for inference and sharing later.

• Rotary position embeddings (RoPE) for long-range context.
• RMSNorm for stable, efficient normalisation.
• SwiGLU feed-forward blocks.
• Grouped-query attention (GQA) for efficient inference.
• Custom BPE tokenizer with a 32,000-token vocabulary.

AQ-5B is bilingual in English and Tamil and treats all educational streams as first-class — the sciences, mathematics, languages, engineering and the arts. It is trained on conceptual knowledge from foundational to advanced, giving downstream tutors a broad, reliable base to reason from.

AQ-5B is a foundation model — a base intended for further training and alignment rather than a finished consumer product. It is built to give downstream tutors like ZyoraByte a broad, reliable base to reason from across every educational stream.