ChatGPT vs DeepSeek：不入虎穴，焉得虎子

2025 年 1 月 28 日（除夕），《中國時報》頭條新聞以 「陸 DeepSeek 爆紅」 為題，報導了這款 AI 模型的崛起。在好奇心與求知慾的驅使下，花了一整天，以科學實證的方式，對 DeepSeek 與 ChatGPT 進行比較與測試，結果顯示兩者在多個層面存在高度相似性。

外觀與使用介面：從介面設計、回應格式到輸入框架，DeepSeek 的視覺與使用體驗幾乎與 ChatGPT 如出一轍，僅有細微差異。

• 使用者介面（UI）：版面設計、提示字樣、回應格式及字體均與 ChatGPT 高度相似，唯一明顯不同的是 Prompt 部分採用了藍色背景。

• 語言風格：DeepSeek 英文回應的措辭（diction and syntax）、表達方式（expression and style）與 ChatGPT 頗為相近，呈現出相似的語感與行文模式。

• 數學公式排版：DeepSeek 同樣採用 LaTeX 進行數學公式排版。

• 回應結尾風格：DeepSeek 亦在回答結尾使用相同的微笑表情符號😊，這在 AI 模型中並不常見。

核心技術與測試實證：為進一步檢驗其技術基礎，透過預訓練（pre-training）與推理（inference）相關概念進行測試，發現DeepSeek 與 ChatGPT 亦極為接近，甚至在關鍵描述與專有名詞上幾乎完全一致。

• 核心技術雷同：測試顯示，DeepSeek 在 self-attention 機制 與 feed-forward neural networks 的敘述方式、使用的術語，幾乎是 ChatGPT 的翻版。

• 模型架構相似：DeepSeek 的回應中提及 GPT-2 及 GPT-3 的技術，例如： 「例如 GPT-2 有 12 層，GPT-3 有 96 層。」此描述原為 OpenAI 早期發佈的 GPT-2、GPT-3 預訓練架構資訊，然而 DeepSeek 在回應中未明確指出這些層數屬於 OpenAI，而是以舉例方式呈現，容易讓人誤解為其自身架構的一部分。

• 範例句完全相同：例如在測試過程中，DeepSeek 使用例句： “The cat sat on the mat.”（貓坐在墊子上） 這與 ChatGPT 的舉例完全相同。

此外，值得關注的是，OpenAI 在 GPT-4 訓練時採用了 H100 GPU，而 DeepSeek 則使用 H800 晶片(H100限制出口)。然而，H800 的整體效能約為 H100 的70%-80%，DeepSeek 卻僅花了 兩個月 即完成模型訓練，這不禁令人好奇其技術突破。

不入虎穴，焉得虎子：科學研究講求＂基於事實（reality-based）＂的驗證。透過一天的測試與比較，DeepSeek 在 self-attention 機制 與 feed-forward neural networks 等關鍵技術上，表現出與 ChatGPT 近乎一致的結果，甚至在回應風格、術語選擇與範例內容上皆高度相似。技術的發展可以趨同，但如果連內容敘述都極為相近，則值得進一步探討。

以下為測試實證：（中英對照，先英文，再譯為中文。建議手機打橫著看，比較清楚）

DeepSeek的Prompt以藍底標示：

DeepSeek 回答：

你好！很高興認識你。是的，注意力機制(attention mechanism)是訓練現代深度學習模型的關鍵部分，尤其是在自然語言處理(Natural Language Processing, NLP)任務中。注意力機制讓模型在進行預測時，能專注於輸入序列的特定部分，這對於機器翻譯(machine translation)、文本摘要(text summarization)等任務特別有用。

其中最受歡迎的注意力機制之一是自注意力機制(self-attention mechanism)，它是Transformer架構(Transformer architecture)的核心組件。自注意力機制允許模型根據序列中不同詞語之間的相對重要性進行加權。此機制被應用於如BERT、GPT等模型中。

以下是自注意力機制運作的高層次概述：

• 輸入表示(Input Representation)：輸入序列首先被轉換為嵌入向量(embeddings)（向量表示）。

•查詢、鍵與值(Query, Key, and Value)：
序列中的每個詞會生成三個向量：查詢向量(Query vector, Q)、鍵向量(Key vector, K) 和 值向量(Value vector, V)。

• 注意力分數(Attention Scores)：
兩個詞之間的注意力分數通過其查詢向量與鍵向量的點積計算。此分數表示在編碼特定詞時，應對序列中其他部分投入多少關注。

• Softmax歸一化(Softmax)：
注意力分數通過softmax函數轉換為注意力權重，歸一化後總和為1。

• 加權求和(Weighted Sum)：
值向量根據注意力權重進行加權並求和，生成每個詞的最終輸出表示。

自注意力機制讓模型能夠捕捉序列中詞語之間的依賴關係，無論它們的距離多遠。這相較於早期的RNN或LSTM模型是一大改進。

若您有更多關於注意力機制的具體問題，或想了解它們在特定模型中的實現方式，歡迎隨時提問！ 😊

---

Hi! Nice to meet you too. Yes, attention mechanisms are a crucial part of training modern deep learning models, especially in natural language processing (NLP) tasks. The attention mechanism allows the model to focus on specific parts of the input sequence when making predictions, which is particularly useful for tasks like machine translation, text summarization, and more.

One of the most popular attention mechanisms is the self-attention mechanism, which is the core component of the Transformer architecture. Self-attention allows the model to weigh the importance of different words in a sequence relative to each other. This mechanism is used in models like BERT, GPT, and others.

Here’s a high-level overview of how self-attention works:

1. Input Representation: The input sequence is first converted into embeddings (vector representations).
2. Query, Key, and Value: For each word in the sequence, three vectors are computed: a Query vector (Q), a Key vector (K), and a Value vector (V).
3. Attention Scores: The attention score between two words is computed by taking the dot product of their Query and Key vectors. This score indicates how much focus to place on other parts of the sequence when encoding a particular word.
4. Softmax: The attention scores are passed through a softmax function to obtain attention weights, which are normalized to sum to 1.
5. Weighted Sum: The Value vectors are weighted by the attention weights and summed to produce the final output for each word.

The self-attention mechanism allows the model to capture dependencies between words regardless of their distance in the sequence, which is a significant improvement over earlier models like RNNs or LSTMs.

If you have any more specific questions about attention mechanisms or how they are implemented in particular models, feel free to ask!

是的，確實如此！在 Transformer 架構中，自注意力機制（self-attention mechanism）之後會接一個前饋神經網絡（Feed-Forward Neural Network, FFN）。這是每個 Transformer 層中的關鍵組件，與自注意力機制協同工作，進一步處理和轉換數據。讓我為您詳細解釋：

---

前饋神經網絡（FFN）的作用：

在自注意力機制計算完輸入序列的加權表示後，輸出會通過一個前饋神經網絡。FFN 會獨立地應用於序列中的每個位置（例如每個詞或 token）。它的目的是進一步處理和轉換數據，增加非線性，使模型能夠學習更複雜的模式。

---

前饋神經網絡的結構：

Transformer 層中的 FFN 通常由兩個線性變換和中間的非線性激活函數組成。以下是它的運作方式：

1.第一個線性層：

•擴展輸入的維度（例如從 512 維擴展到 2048 維）。

•這使得模型能夠捕捉更複雜的特徵。

2.激活函數：

應用一個非線性激活函數（通常是 ReLU 或 GELU）來引入非線性。

這幫助模型學習更複雜的表示。

3.第二個線性層：

•將維度縮減回原始大小（例如從 2048 維縮減到 512 維）。

•這確保輸出可以傳遞到下一層或用於後續步驟。

數學上，FFN 可以表示為：

---

為什麼 FFN 很重要：

增加非線性：自注意力機制主要是線性的（除了 softmax 函數）。FFN 引入了非線性，使模型能夠學習更複雜的函數。

逐位置處理：FFN 獨立地應用於序列中的每個位置，使模型能夠分別處理每個 token 的表示，同時仍然受益於自注意力捕捉的上下文。

特徵轉換：FFN 將注意力輸出轉換為對下一層或任務（例如分類、生成）更有用的表示。

---

它在 Transformer 層中的位置：

一個 Transformer 層通常包括：

•　多頭自注意力（Multi-Head Self-Attention）：捕捉序列中 token 之間的關係。

•　添加與歸一化（殘差連接 + 層歸一化）：穩定訓練並改善梯度流動。

•　前饋神經網絡（FFN）：進一步處理數據。

•　另一個添加與歸一化：確保梯度傳播順暢。

這種自注意力和 FFN 的組合在多個層中重複（例如 GPT-2 有 12 層，GPT-3 有 96 層），使模型能夠學習越來越抽象和複雜的輸入數據表示。

---

實際範例：

如果輸入是一個句子，例如「貓坐在墊子上」，自注意力機制幫助模型理解「貓」與「坐」和「墊子」之間的關係。FFN 則將這些上下文相關的表示轉換為對下一層或最終任務（例如預測下一個詞或分類句子）更有用的形式。

---

如果您想進一步了解 FFN 如何與其他組件互動，或者它在特定任務中的作用，請隨時告訴我！😊

Yes, absolutely! Following the self-attention mechanism in the Transformer architecture, there is indeed a feed-forward neural network (FFN). This is a critical component of each Transformer layer and works in conjunction with the self-attention mechanism to process and transform the data. Let me break it down for you:

---

Role of the Feed-Forward Neural Network (FFN):

After the self-attention mechanism computes the attention-weighted representations of the input sequence, the output is passed through a feed-forward neural network. The FFN is applied independently to each position (e.g., each word or token) in the sequence. Its purpose is to further process and transform the data, adding non-linearity and enabling the model to learn more complex patterns.

---

Structure of the Feed-Forward Neural Network:

The FFN in a Transformer layer typically consists of two linear transformations with a non-linear activation function in between. Here’s how it works:

1. First Linear Layer:
   1. Expands the dimensionality of the input (e.g., from 512 to 2048 dimensions).
   1. This allows the model to capture more complex features.
2. Activation Function:
   1. A non-linear activation function (usually ReLU or GELU) is applied to introduce non-linearity.
   1. This helps the model learn more sophisticated representations.
3. Second Linear Layer:
   1. Reduces the dimensionality back to the original size (e.g., from 2048 to 512 dimensions).
   1. This ensures the output can be passed to the next layer or used in subsequent steps.

Mathematically, the FFN can be expressed as:

Why the FFN is Important:

1. Adds Non-Linearity:
   The self-attention mechanism is primarily linear in nature (aside from the softmax function). The FFN introduces non-linearity, enabling the model to learn more complex functions.
2. Position-Wise Processing:
   The FFN is applied independently to each position in the sequence, allowing the model to process each token’s representation separately while still benefiting from the context captured by self-attention.
3. Feature Transformation:
   The FFN transforms the attention outputs into a more useful representation for the next layer or task (e.g., classification, generation).

---

How It Fits into the Transformer Layer:

A single Transformer layer typically consists of:

1. Multi-Head Self-Attention: Captures relationships between tokens in the sequence.
2. Add & Normalize (Residual Connection + Layer Normalization): Stabilizes training and improves gradient flow.
3. Feed-Forward Neural Network: Further processes the data.
4. Another Add & Normalize: Ensures smooth gradient propagation.

This combination of self-attention and FFN is repeated across multiple layers (e.g., 12 layers in GPT-2, 96 layers in GPT-3), allowing the model to learn increasingly abstract and sophisticated representations of the input data.

---

Example in Practice:

If the input is a sentence like “The cat sat on the mat,” the self-attention mechanism helps the model understand how “cat” relates to “sat” and “mat.” The FFN then takes these contextualized representations and transforms them into a form that’s useful for the next layer or the final task (e.g., predicting the next word or classifying the sentence).

---

If you’d like to dive deeper into how the FFN interacts with other components or its role in specific tasks, feel free to ask! 😊

English Edition for International Readers in Taiwan

ChatGPT vs. DeepSeek: No Risk, No Reward

On January 28, 2025 (Lunar New Year’s Eve), China Times ran a front-page article titled “DeepSeek Booms in China”, reporting on the rise of this AI model. Driven by curiosity and a thirst for knowledge, I spent an entire day conducting a scientific and evidence-based comparison between DeepSeek and ChatGPT. The results revealed a high degree of similarity between the two models across multiple aspects.

---

1. Appearance and User Interface

From interface design to response formatting and input structure, DeepSeek’s visual and user experience are nearly identical to ChatGPT’s, with only minor differences:

• User Interface (UI): The layout, prompt display, response format, and fonts are highly similar to ChatGPT, with the only notable difference being that the prompt section in DeepSeek features a blue background.
• Linguistic Style: DeepSeek’s diction and syntax, as well as its expression and style, closely resemble those of ChatGPT, producing a similar tone and writing style.
• Mathematical Formatting: DeepSeek also employs LaTeX for mathematical equation formatting.
• Response Endings: Like ChatGPT, DeepSeek concludes responses with the same smiling emoji 😊, which is uncommon among AI models.

---

2. Core Technology and Empirical Testing

To further examine its technological foundation, I tested DeepSeek using concepts related to pre-training and inference. The findings indicate that DeepSeek’s responses closely align with ChatGPT’s, even in key descriptions and specialized terminology.

• Highly Similar Core Technologies: Tests show that DeepSeek’s explanations of the self-attention mechanism and feed-forward neural networks—including the terminology used—are nearly identical to those of ChatGPT.
• Comparable Model Architecture: In its responses, DeepSeek referenced GPT-2 and GPT-3’s architecture, stating:

“For example, GPT-2 has 12 layers, while GPT-3 has 96 layers.”
This description originally comes from OpenAI’s published details on the pre-training architecture of GPT-2 and GPT-3. However, DeepSeek did not explicitly clarify that these layer counts belong to OpenAI, instead presenting them as examples, which could easily mislead users into believing that this is part of its own architecture.

• Identical Example Sentences: During testing, DeepSeek provided the example:

“The cat sat on the mat.” (貓坐在墊子上)
This exact phrase has been widely used in ChatGPT’s prior demonstrations, raising further questions about the extent of their similarity.

Additionally, it is noteworthy that OpenAI used H100 GPUs to train GPT-4, whereas DeepSeek reportedly used H10 chips. Given that H10’s computing power and memory bandwidth are roughly half of H100’s, it is intriguing that DeepSeek managed to complete its model training within just two months—a feat that prompts curiosity about its technological advancements.

---

3. No Risk, No Reward

Scientific research is based on reality-based validation. Through a full day of testing and comparison, DeepSeek demonstrated results that were nearly identical to ChatGPT’s in key areas, including self-attention mechanisms, feed-forward neural networks, response style, terminology usage, and even example content.

While technological developments tend to converge over time, when even content structuring and wording become highly similar, further discussion is warranted.