Struct LocalEngine 

pub struct LocalEngine { /* private fields */ }

A model loaded in memory. Loading takes a while and generating blocks, so keep it away from the UI thread.

Implementations

impl LocalEngine

pub fn load( folder: &ModelFolder, model: &LocalModel, ) -> Result<Self, EngineError>

Loads model from folder.

Errors

When its files are missing or cannot be read.

pub fn generate( &mut self, system: &str, user: &str, max_tokens: usize, on_text: &mut dyn FnMut(&str) -> bool, ) -> Result<Generated, EngineError>

Answers user, following system, in at most max_tokens. on_text gets the text as it comes, and returning false stops it.

Errors

When it stops in the middle.

Trait Implementations

impl Debug for LocalEngine

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.