Some communities want their users to have access to Large Language Models (LLMs) / Generative AI (GenAI) models. For example, via coding agents in the terminal or Jupyter AI in JupyterLab. This is a new and evolving space, so your best bet is to learn how other communities have set up their hubs and workflows, and look to their configuration for inspiration.
For one approach, check out our blog post: How we set up an AI-enabled hub for the Responsible GenAI workshop. It covers the configuration used by the CryoCloud and NASA ESDS communities for a recent GenAI workshop, including the user image, model access, and what we’d improve next time.
A few things to keep in mind:
Install AI tools in your community image, like any other package.
Try AI tools in a separate image and test it out before you do anything on your main “production” image. Jupyter AI can still be a bit unstable (there’s some upstream work happening around this, see: jupyter
-ai -contrib /jupyter -server -documents #289 for example). Model providers need an API key. 2i2c can add a shared key to your hub, but every user can read it.
Reach out to support if you’d like to set up something similar.