Large Concept Models: Language Modeling in a Sentence Representation Space

Der Steppenwolf
  2025-03-17 09:41:03 2025-03-17 09:41:03 Created   2025-06-22 20:46:50 2025-06-22 20:46:50 Updated    

Barrault, Loïc, Paul-Ambroise Duquenne, Maha Elbayad, Artyom Kozhevnikov, Belen Alastruey, Pierre Andrews, Mariano Coria et al. “Large Concept Models: Language Modeling in a Sentence Representation Space.” arXiv e-prints (2024): arXiv-2412.

What problem does this paper address?

What is the motivation of this paper?

What are the main contributions?

Which insights inspired the proposed model architecture?

How does the SONAR text embedding space work?

How is the data pre-processed?

Model Architectures

Base-LCM

Diffusion-based LCM

One-Tower Diffusion LCM

Two-Tower Diffusion LCM

Quantized LCM

Experiments

Results and Conclusions

What are the strengths and weaknesses of this paper?

What insights does this paper provide?

  • Title: Large Concept Models: Language Modeling in a Sentence Representation Space
  • Author: Der Steppenwolf
  • Created at : 2025-03-17 09:41:03
  • Updated at : 2025-06-22 20:46:50
  • Link: https://st143575.github.io/steppenwolf.github.io/2025/03/17/Large-Concept-Models/
  • License: This work is licensed under CC BY-NC-SA 4.0.
Comments
On this page
Large Concept Models: Language Modeling in a Sentence Representation Space
  1. What problem does this paper address?
  2. What is the motivation of this paper?
  3. What are the main contributions?
  4. Which insights inspired the proposed model architecture?
  5. How does the SONAR text embedding space work?
  6. How is the data pre-processed?
  7. Model Architectures
    1. Base-LCM
    2. Diffusion-based LCM
    3. Quantized LCM
  8. Experiments
  9. Results and Conclusions
  10. What are the strengths and weaknesses of this paper?
  11. What insights does this paper provide?