Current Status

Currently ollama models are functional, and accessible from the otdfctl chat command.

Configurations are managed in a chat_config.json file located in the home directory, and are being loaded in via the chat_config.go file. This is temporary as there is certainly a more graceful way of storing and managing chat parameters. Currently the parameters are:

{
    "model": "llama3",
    "verbose": true,
    "apiURL": "http://localhost:11434/api/generate"
}

This could technically be model agnostic so long as it runs on the same port and url, and has the same REST-like structure for handling queries that ollama supports. Verbosity could also be changed to a string ("high", "med", "low") but it is simpler to start with a bool.

Currently, verbose controls if the entire sanitized prompt is shown to the user before a response. This can and should include much more especially during initial startup.

TODOs:

• Graceful startup in initial loading of configs using chat_config.go
• Graceful exits and additional error checks for if the model is not running or if there are other trivial issues
• Test secondary model and refine configurations to make implementation more model-agnostic (Gemma, TinyChatEngine)
• Organize sanitization prompts for different levels of user familiarity
• Collect and disseminate generalized Q&As to quality-test our prompt engineering efforts (vibe-check the model and prompting for our use-case)
• Open-ended: Investigate improved prompt engineering efforts for ollama models
• Open-ended: Benchmark both 'performance' as well as speed between model types but more explicitly on prompt engineering efforts

Links to explore for improved prompt engineering:
llama2 prompt engineering
How to prompt llama3