Foundation model

A foundation model is a large machine learning model trained on broad, wide-ranging data, intended to serve as a base that can be adapted (through further training or prompting) to many specific downstream tasks, rather than being built from scratch for one narrow purpose.¹ Large language models are the most widely known example, though the concept extends to other data types, including, as an active research direction, geometry.

Why adapting a foundation model is cheaper than training from scratch

Adapting an existing foundation model to a new task is typically far cheaper than training a comparable model from scratch, which is the main practical appeal of the foundation-model approach, though the foundation model's own broad training still requires substantial data and compute.

Where foundation models apply

Not on the drawing; a foundation model is a category of machine learning model used within software, not something specified by a part's requirements.

Common mistakes

Assuming any large machine learning model qualifies as a foundation model; the term specifically implies broad training intended for adaptation to many downstream tasks, not just large scale on its own.

Is a large language model the same as a foundation model?

A large language model is one prominent example of a foundation model, specifically for text; the broader concept extends to other data types as well.

Does a geometry foundation model already exist?

Not as an established, broadly validated technology; it remains an emerging research direction, distinct from the well-established language foundation model concept.

Why adapt a foundation model instead of training from scratch?

Because adapting an existing broadly trained model to a new task is typically far cheaper in data and compute than training a comparable model from nothing.