lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled
Text Generation • 36B • Updated • 770 • 17
Pavel AI
pavel-ai
AI & ML interests
AI tools development
Recent Activity
liked a model about 18 hours ago
lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled liked a model 4 days ago
nlpaueb/legal-bert-base-uncased liked a model 12 days ago
google/gemma-4-31B-itOrganizations
liked a
model about 18 hours ago
liked a
model 4 days ago
liked a
model 12 days ago
upvoted 2
collections 18 days ago
liked 3
models 25 days ago
liked a
Space about 1 month ago
liked a
model about 1 month ago
liked 2
Spaces 3 months ago
upvoted an article 3 months ago
Article
RexRerankers: SOTA Rankers for Product Discovery and AI Assistants
•
44
updated a
dataset 3 months ago
published a
dataset 3 months ago
liked a
Space 3 months ago
commented on Reverse Engineering a $500M Mystery: From HashHop to Memory-Augmented Language Models 3 months ago
Does it mean that the Qwen model is passing tokens back to the context. In the retrieval level these tokens got encoded into MALM model's hashes which are used to retrieve information from context in MALM tokens and then the result converts to Qwen's tokens again?
reacted to codelion's post with 🔥 3 months ago
Post
3249
Reverse Engineering a $500M Mystery: From HashHop to Memory-Augmented Language Models
I wrote a deep dive into how Magic AI's 100M token context window might work, starting from their HashHop benchmark and building up to MALM - a Memory-Augmented Language Model.
Key insight: treating each key as a single token enables perfect retrieval at unlimited context lengths.
The article covers:
- How HashHop works and why its perfect accuracy is suspicious
- Building a tokenized solver that achieves 100% accuracy
- Scaling to MALM for real code search tasks
- Why this approach could handle 100M+ tokens
Read the full article: https://huggingface.co/blog/codelion/reverse-engineering-magic-hashhop
Try the model: codelion/malm-165m
Code: https://github.com/codelion/hash-hop
I wrote a deep dive into how Magic AI's 100M token context window might work, starting from their HashHop benchmark and building up to MALM - a Memory-Augmented Language Model.
Key insight: treating each key as a single token enables perfect retrieval at unlimited context lengths.
The article covers:
- How HashHop works and why its perfect accuracy is suspicious
- Building a tokenized solver that achieves 100% accuracy
- Scaling to MALM for real code search tasks
- Why this approach could handle 100M+ tokens
Read the full article: https://huggingface.co/blog/codelion/reverse-engineering-magic-hashhop
Try the model: codelion/malm-165m
Code: https://github.com/codelion/hash-hop
- 1 reply
upvoted an article 3 months ago
Article
Reverse Engineering a $500M Mystery: From HashHop to Memory-Augmented Language Models
•
10
liked a
model 3 months ago
