This document is published in Chinese and English — use the language switch in the header. A PDF edition is available in both languages, and the download button follows the current language. Document no. IRIXEN-WP-2026-001, version 1.0, prepared by IRIXEN Team.
Abstract
IRIXEN-Embedding-V2 is a parameter-efficient multi-task text embedding model. On top of a frozen Qwen3-Embedding-4B (4.02B parameters) it trains only 68.8M parameters — 1.68% of the full model: a low-rank adapter over the q/k/v/o projections (LoRA, r=64, α=128), plus a set of product-manifold geometry heads that map the representation into different geometric spaces: a sphere S¹⁰²⁴, two hyperbolic heads L¹ (H⁵¹², c=0.5) and L² (H⁵¹², c=2.0), and a Euclidean head R¹⁰²⁴.
The model wins five of six benchmarks: MIRACL retrieval 95.10% (+3.65 over the base), cross-lingual retrieval on four unseen languages 93.90% (+5.35), T2 Chinese reranking 82.87% (+11.07), iFLYTEK 119-class classification 64.05% (+15.70) and zero-shot CLS clustering 76.94% (+1.86). On STS, which is not part of training, it reaches 86.61% — 7.09 points below the base.
The conclusion comes first: the gains come from the geometry of the representation space, not from stacking parameters. Training was 1,000 base steps plus 800 classifier-head steps, about 90 minutes on one MI300X, and bf16 inference needs roughly 10 GB. We do not paper over the STS gap; section 6.1 gives a routing scheme that can be deployed as is.
1. Overview
1.1 Positioning
IRIXEN-Embedding-V2 is a parameter-efficient multi-task text embedding model. Built on a frozen Qwen3-Embedding-4B base, it combines low-rank adaptation (LoRA) with product-manifold geometry heads so that a single model learns a unified representation for retrieval, reranking, classification, clustering and similarity.
The full model is 4.02B parameters (frozen base); new trainable parameters are 68.8M, or 1.68% of the whole. bf16 inference needs about 10 GB, and all training completes in roughly 90 minutes on a single AMD MI300X.
1.2 Core thesis
Different semantic tasks need different geometric spaces. Forcing retrieval, classification, clustering and similarity into one Euclidean space is the hidden bottleneck of current embedding models. IRIXEN-Embedding-V2 uses a product-manifold architecture so that each task lands in the space that matches its intrinsic structure — improving representation quality at the architectural level rather than through parameter tuning.
1.3 Key metrics
Table 1-1 · Key metrics
Metric
Value
Note
Base model
Qwen3-Embedding-4B (4.02B)
Frozen for training and inference
Trainable parameters
68.8M
1.68% of the full model
Inference memory
~10 GB
bf16
Training hardware
AMD MI300X 192GB × 1
All training on one card
Total training time
~90 minutes
Including classifier refinement
MIRACL retrieval (10 languages)
95.10%
+3.65 points over base
MIRACL cross-lingual (4 unseen)
93.90%
+5.35 points over base
iFLYTEK classification (119 classes)
64.05%
+15.70 points over base
1.4 Terms and abbreviations
Table 1-2 · Terms and abbreviations
Term
Meaning
Text embedding
Mapping text to a fixed-dimensional dense vector for retrieval, classification and other downstream computation
LoRA
Low-Rank Adaptation — parameter-efficient fine-tuning that trains a small number of incremental parameters
Product manifold
A representation space formed as the direct product of spaces with different geometry (sphere, hyperbolic, Euclidean)
Hyperbolic space H
A negatively curved space whose volume grows exponentially with radius, a natural fit for tree-like hierarchies
Lorentz model
One realisation of hyperbolic space, with curvature c controlling how strongly the space bends
Multi-head pooling
Aggregating attention pooling, last-token pooling and mean pooling with a learnable gate
Rerank
Re-ordering first-stage candidates by relevance
MIRACL
A multilingual retrieval benchmark
STS
Semantic Textual Similarity, measured by Spearman correlation
Task head: RegClassifier, taking the 1026-dimensional concatenation of the L¹ and L² hyperbolic features and emitting 119 classes.
2.2 Data flow
The end-to-end data flow is shown below. The base and the adapter only encode tokens into contextual representations; every task difference is handled by the pooling fusion and the geometry heads.
Figure 2-1 · End-to-end data flow: shared encoder plus per-task geometry heads
2.3 Parameter breakdown
Table 2-1 · Parameter breakdown
Component
Parameters
Note
Qwen3-Embedding-4B
4,021,762,816
Frozen, not trained
LoRA (r=64, α=128)
47,185,920
On q/k/v/o projections
MultiHeadPooling
2,563
Attention query and fusion gate
SphericalHead
7,342,080
2560→2048→1024, two-layer MLP
LorentzHead-1
3,146,240
2560→1024→512, curvature c=0.5
LorentzHead-2
3,146,240
2560→1024→512, curvature c=2.0
EuclideanHead
7,342,080
2560→2048→1024, two-layer MLP
RegClassifier
589,000
LayerNorm + Dropout(0.3) + 1026→512→119
Total trainable
68.8M
1.68% of the full model
3. Core techniques
3.1 Product-manifold architecture
Conventional embedding models map semantics into a single Euclidean space R^d. IRIXEN-Embedding-V2 uses a product manifold as the unified representation space:
M = S¹⁰²⁴ × H⁵¹²(c=0.5) × H⁵¹²(c=2.0) × R¹⁰²⁴
The spherical component carries retrieval and reranking, the two hyperbolic components with different curvature model coarse and fine hierarchy respectively, and the Euclidean component carries continuous similarity. The geometry and task fit of each component:
Table 3-1 · Geometry and task fit of each component
Why two curvatures in parallel: hyperbolic volume grows exponentially with radius and therefore hosts tree-like hierarchies naturally. Shallow categories (e.g. "tech / finance") suit low-curvature space, while deep sub-categories (e.g. "AI chips / EVs") suit high-curvature space. Modelling both with parallel hyperbolic heads of different curvature is the structural contribution of IRIXEN-Embedding-V2.
3.2 Multi-head pooling
Rather than relying on last-token or mean pooling alone, the model pools three ways and fuses them with a learnable gate:
pooled = g₁ · AttentionPool(h) + g₂ · LastToken(h) + g₃ · MeanPool(h)
where g = softmax(learnable_gate), updated during training
This resolves the mismatch in pooling preference across tasks: retrieval favours last-token pooling, classification favours attention pooling, and STS-style tasks favour mean pooling. The gate lets one model converge on the right mixture per task.
3.3 Two-stage classifier refinement
For classification (iFLYTEK, 119 classes) the model is trained in two stages:
Stage one: jointly train the base LoRA, the hyperbolic heads and the classifier.
Stage two: freeze the LoRA and hyperbolic heads, retrain only the classifier, and add LayerNorm, Dropout(0.3) and label smoothing (0.1).
Stage two matters: the train-validation gap narrows from +7.9% to +4.6%, and test accuracy improves by 2.55 points.
4. Training
4.1 Data
Table 4-1 · Training and validation data
Dataset
Size
Languages
Use
MIRACL train
28,630 triplets
zh / en / es / fr / de / ru / ja / ko / ar / it (10)
Retrieval training
MIRACL dev
11,452 triplets
hi / th / vi / tr (4 unseen)
Cross-lingual validation
T2 reranking
17,047 triplets
Chinese
Reranking training
iFLYTEK
24,649 samples
Chinese, 119 classes
Classification training
4.2 Losses
Retrieval uses a margin triplet loss:
L = relu(0.05 + ⟨q, n⟩ − ⟨q, p⟩)
where q is the query, p the positive and n the negative. Classification uses label-smoothed cross-entropy:
L = CrossEntropy(Classifier(LorentzFeature), y)
label_smoothing = 0.1
bf16 for the base, fp32 for geometry heads and classifier
Gradient clipping
1.0
Hardware
AMD MI300X 192GB, single card
Total time
~90 minutes
5. Evaluation
5.1 Protocol
All comparisons use the same clean data, the same tokenizer and the same sampling method. The baseline Qwen3-Embedding-4B uses its official best instructions and IRIXEN-Embedding-V2 the corresponding manifold output, so the comparison reflects structural differences only — not prompt or data differences.
5.2 Results
Table 5-1 · Comparison with the base model (5 wins, 1 loss)
Task
IRIXEN-Embedding-V2
Qwen3-Emb-4B (base)
Delta
MIRACL retrieval (10 languages)
95.10%
91.45%
+3.65
MIRACL cross-lingual (4 unseen)
93.90%
88.55%
+5.35
T2 Chinese reranking
82.87%
71.80%
+11.07
iFLYTEK classification (119 classes)
64.05%
48.35%
+15.70
Zero-shot CLS clustering
76.94%
75.08%
+1.86
STS Spearman (not trained)
86.61%
93.70%
−7.09
5.3 Reading the results
Classification leads by 15.70 points. The advantage of hyperbolic geometry over hierarchies follows from volume growing exponentially with radius — a structural property, not a tuning result.
Cross-lingual generalisation reaches 93.90%. Training used 10 languages; validation used four that never appeared in training (Hindi, Thai, Vietnamese, Turkish), which indicates the learned direction space is a language-independent semantic representation.
STS is not trained and we say so. The model trails the base by 7.09 points on this task; in production, route such requests to the Qwen3-Embedding-4B base with the STS instruction (up to 93.70%).
6. Deployment guidance
6.1 Task routing
Given the results in section 5, production deployments should route by task. One set of weights covers four of the five request types; only STS falls back to the instruction-tuned base.
Table 6-1 · Task routing
Task
Model and head
Reference metric
Retrieval
IRIXEN-Embedding-V2 (spherical head)
95.10% (cross-lingual 93.90%)
Rerank
IRIXEN-Embedding-V2 (spherical head)
82.87%
Classification
IRIXEN-Embedding-V2 (classifier)
64.05%
Clustering
IRIXEN-Embedding-V2 (Lorentz head)
76.94%
Similarity (STS)
Qwen3-Embedding-4B base + STS instruction
93.70%
6.2 Runtime requirements
Table 6-2 · Runtime requirements
Item
Requirement
Inference memory
~10 GB (bf16)
Batch size
32 recommended
First load
~30 seconds
Attention
sdpa recommended (best ROCm compatibility)
6.3 Inference API
A standard Python interface is provided. Retrieval and classification share the same weights and diverge only at the output head:
from inference import IRIXEN2
m = IRIXEN2(base_path="...", model_dir="./IRIXEN-Embedding-V2")
# Retrieval: encode queries and documents, dot product is the score
eq = m.encode(["query text"], task="retrieval")
ed = m.encode(["document text"], task="retrieval")
sim = eq @ ed.T
# Classification: returns labels directly
labels = m.classify(["app description text"])
Figure 6-1 · Calling conventions for retrieval and classification
6.4 Compatibility notes
On AMD ROCm, use attn_implementation="sdpa"; FlashAttention 2 can hang during inference on ROCm (confirmed in development).
flash-attn 2.8.3 is an optional dependency, not a required component.
7. Known limitations
So that users can choose correctly, the known limitations are disclosed in full:
STS is not trained. The model contains no STS training and trails the base on that task; route such traffic to the base model per section 6.1.
The classifier is language-bound. The iFLYTEK classifier handles Chinese topic classification only; other languages require retraining the head.
Training is not saturated. The run is 1,000 base steps plus 800 classifier steps; there is headroom. Following the current trend, iFLYTEK accuracy is expected to exceed 65%.
ROCm constraints. Choose the attention implementation per section 6.4; FlashAttention 2 is not usable on ROCm.
Model evaluation, selection and production deployment reference
Every metric here is reproducible on the same clean data, tokenizer and sampling method. To check the evaluation scripts and sampling manifests, contact us through the site ticket form.
