Скачать или смотреть LLMs and The Probability of the Next Word

This video gives an overview of what the "probability of the next word" really means when it comes to LLMs.

While precise details about the architecture and size of large language models like GPT-4 are often kept confidential by the developing companies (in this case, OpenAI), there have been various estimations and reports.

It's widely speculated that GPT-4 has a significantly larger number of parameters than its predecessor, GPT-3. GPT-3 had 175 billion parameters. Guesses for GPT-4 have ranged from around 1 trillion to even exceeding 100 trillion parameters.

However, it's important to understand a few things:

Parameter count isn't the only metric of performance: While a larger number of parameters generally allows a model to learn more complex patterns, other factors like training data quality, architecture, and training methods also play crucial roles.
Official confirmation is lacking: OpenAI has not publicly disclosed the exact number of parameters for GPT-4. Therefore, any figures you encounter should be treated as estimates or rumors.
In summary, it's highly likely that GPT-4 has a substantially larger number of parameters than GPT-3, potentially in the trillions, but the precise figure remains undisclosed.