| Contents |
Since the success of large language models, Foundation Model has attracted a great attention. Foundation Model aims to provide a pre-trained model, typically a large deep learning model, that can be easily fine-tuned or re-purposed for a downstream task with a smaller data set. Where there has been much progress in the language and vision domains, Foundation Model for time series data is not relatively well studied. Here, we propose a preliminary Foundation Model targeted to nonlinear dynamical systems. We consider the problem of building a foundation model from a range of dynamical systems and generating a time series data from the foundation model given a short sequence of input data. The foundation model consists of two modules. In the first module, we aim to learn a dictionary of functions. We propose to use a contrastive learning with a hierarchy to build a dictionary of different classes of dynamical systems. Then, we use a denoising diffusion model to generate the time series data from the dictionary. The foundation model can also be used for a forecast of the input sequence. |