Small language model
Small language model are artificial intelligence language model designed for human natural language processing including language and text generation. Unlike large language models, small language model are much more smaller in scale and scope.
Typically, a large language model's training parameter number in the hundreds of billions, with some models even exceeding a trillion parameters. The scale large language model is so vast that it contains a huge amount of information, allowing it to generate better content. However, this requires enormous computational power, making it impossible for an individual to train a large language model using just a single or few computer and GPU.
Small language model, on the other hand, use far fewer parameters, typically ranging from a few million to a few billion. This make them more feasible to train and host in a resource constrained environments such as a single computer or even a mobile device. [1][2][3][4]
References
- ^ Rina Diane Caballar. "What are small language models?". IBM.
- ^ John JOhnson. "Small Language Models (SLM): A Comprehensive Overview". Huggingface.
- ^ Kate Whiting. "What is a small language model and how can businesses leverage this AI tool?". The World Economic Forum.
- ^ "SLM (Small Language Model) with your Data". Microsoft.