Hi there, this post is for sharing the review of AI Research paper focused on Latent Reasoning. This post is consist of serveral article and each article deals one research paper. You can easily go to the article by hyperlink table.

Latent Reasoning
• Training Large Language Models to Reason in a Continuous Latent Space

§ Latent Reasoning

Before jumping into Latent Reasoning, you should know about Chaing-of-Thought(CoT), of course how the LLM works. But don’t worry! I will explain what they are in simple way in this article. If you want to know about it in more detail, please see another article.

· LLM Mechanism
Most of Large Language Models(LLM) follow the transformer architecture. We can roughly classify transformer as two type, Encoder and Decoder. Encoder is for making high quality words or sentence representation as a vector. In contrast, decoder generates a sentence with auto-regressive way. It requires huge pages to deal all about it.. So let me focus only on the Decoder Architecture, which is the target model in Latent Reasoning.

· Decoder Architecture
I think you have a experience using generative AI model such as Chat GPT, Gemini. Those model is based on the Transformer Decoder Architecture. To make you simply and clearly understand Decoder Architecture, I want you to remember two-keywords in Decoder Architecture.

• Self-Attention
• Auto-Regressive

To make high quality sentence, we must consider context information from given text.

· Chaing of Thought