Chat with AI in RStudio

May 5, 2024

chattr is a package deal that allows interplay with Giant Language Fashions (LLMs),
reminiscent of GitHub Copilot Chat, and OpenAI’s GPT 3.5 and 4. The primary car is a
Shiny app that runs contained in the RStudio IDE. Right here is an instance of what it appears to be like
like working contained in the Viewer pane:

Screenshot of the chattr Shiny app, which displays an example of a single interaction with the OpenAI GPT model. I asked for an example of a simple example of a ggplot2, and it returned an example using geom_point()

Determine 1: chattr’s Shiny app

Regardless that this text highlights chattr’s integration with the RStudio IDE,
it’s price mentioning that it really works outdoors RStudio, for instance the terminal.

Getting began

To get began, set up the package deal from CRAN, after which name the Shiny app
utilizing the chattr_app() operate:

# Set up from CRAN
set up.packages("chattr")

# Run the app
chattr::chattr_app()

#> ── chattr - Obtainable fashions 
#> Choose the variety of the mannequin you want to use:
#>
#> 1: GitHub - Copilot Chat -  (copilot) 
#>
#> 2: OpenAI - Chat Completions - gpt-3.5-turbo (gpt35) 
#>
#> 3: OpenAI - Chat Completions - gpt-4 (gpt4) 
#>
#> 4: LlamaGPT - ~/ggml-gpt4all-j-v1.3-groovy.bin (llamagpt) 
#>
#>
#> Choice:
>

After you choose the mannequin you want to work together with, the app will open. The
following screenshot supplies an outline of the totally different buttons and
keyboard shortcuts you should utilize with the app:

Screenshot of the chattr Shiny app top portion. The image has several arrows highlighting the different buttons, such as Settings, Copy to Clipboard, and Copy to new script

Determine 2: chattr’s UI

You can begin writing your requests in the primary textual content field on the prime left of the
app. Then submit your query by both clicking on the ‘Submit’ button, or
by urgent Shift+Enter.

chattr parses the output of the LLM, and shows the code inside chunks. It
additionally locations three buttons on the prime of every chunk. One to repeat the code to the
clipboard, the opposite to repeat it on to your lively script in RStudio, and
one to repeat the code to a brand new script. To shut the app, press the ‘Escape’ key.

Urgent the ‘Settings’ button will open the defaults that the chat session
is utilizing. These could be modified as you see match. The ‘Immediate’ textual content field is
the extra textual content being despatched to the LLM as a part of your query.

Screenshot of the chattr Shiny app Settings page. It shows the Prompt, Max Data Frames, Max Data Files text boxes, and the 'Include chat history' check box

Determine 3: chattr’s UI – Settings web page

Customized setup

chattr will try to determine which fashions you might have setup,
and can embrace solely these within the choice menu. For Copilot and OpenAI,
chattr confirms that there’s an accessible authentication token to be able to
show them within the menu. For instance, when you have solely have
OpenAI setup, then the immediate will look one thing like this:

chattr::chattr_app()
#> ── chattr - Obtainable fashions 
#> Choose the variety of the mannequin you want to use:
#>
#> 2: OpenAI - Chat Completions - gpt-3.5-turbo (gpt35) 
#>
#> 3: OpenAI - Chat Completions - gpt-4 (gpt4) 
#>
#> Choice:
>

library(chattr)
chattr_use("gpt4")
chattr_app()

You can too choose a mannequin by setting the CHATTR_USE atmosphere
variable.

Superior customization

It’s doable to customise many features of your interplay with the LLM. To do
this, use the chattr_defaults() operate. This operate shows and units the
further immediate despatched to the LLM, the mannequin for use, determines if the
historical past of the chat is to be despatched to the LLM, and mannequin particular arguments.

For instance, you might want to change the utmost variety of tokens used per response,
for OpenAI you should utilize this:

# Default for max_tokens is 1,000
library(chattr)
chattr_use("gpt4")
chattr_defaults(model_arguments = listing("max_tokens" = 100))
#> 
#> ── chattr ──────────────────────────────────────────────────────────────────────
#> 
#> ── Defaults for: Default ──
#> 
#> ── Immediate:
#> • {{readLines(system.file('immediate/base.txt', package deal = 'chattr'))}}
#> 
#> ── Mannequin
#> • Supplier: OpenAI - Chat Completions
#> • Path/URL: https://api.openai.com/v1/chat/completions
#> • Mannequin: gpt-4
#> • Label: GPT 4 (OpenAI)
#> 
#> ── Mannequin Arguments:
#> • max_tokens: 100
#> • temperature: 0.01
#> • stream: TRUE
#> 
#> ── Context:
#> Max Information Recordsdata: 0
#> Max Information Frames: 0
#> ✔ Chat Historical past
#> ✖ Doc contents

Modify immediate enhancements

Past the app

Along with the Shiny app, chattr provides a few different methods to work together
with the LLM:

Use the chattr() operate
Spotlight a query in your script, and use it as your immediate

> chattr("how do I take away the legend from a ggplot?")
#> You'll be able to take away the legend from a ggplot by including 
#> `theme(legend.place = "none")` to your ggplot code.

right here.

RStudio Add-ins

chattr comes with two RStudio add-ins:

Screenshot of the chattr addins in RStudio

Determine 4: chattr add-ins

You’ll be able to bind these add-in calls to keyboard shortcuts, making it straightforward to open the app with out having to write down
the command each time. To learn to do this, see the Keyboard Shortcut part within the
chattr official web site.

Works with native LLMs

Open-source, skilled fashions, which can be in a position to run in your laptop computer are broadly
accessible at present. As an alternative of integrating with every mannequin individually, chattr
works with LlamaGPTJ-chat. This can be a light-weight software that communicates
with quite a lot of native fashions. Right now, LlamaGPTJ-chat integrates with the
following households of fashions:

GPT-J (ggml and gpt4all fashions)
LLaMA (ggml Vicuna fashions from Meta)
Mosaic Pretrained Transformers (MPT)

LlamaGPTJ-chat works proper off the terminal. chattr integrates with the
software by beginning an ‘hidden’ terminal session. There it initializes the
chosen mannequin, and makes it accessible to begin chatting with it.

To get began, you want to set up LlamaGPTJ-chat, and obtain a suitable
mannequin. Extra detailed directions are discovered
right here.

chattr appears to be like for the placement of the LlamaGPTJ-chat, and the put in mannequin
in a selected folder location in your machine. In case your set up paths do
not match the places anticipated by chattr, then the LlamaGPT won’t present
up within the menu. However that’s OK, you may nonetheless entry it with chattr_use():

library(chattr)
chattr_use(
  "llamagpt",   
  path = "[path to compiled program]",
  mannequin = "[path to model]"
  )
#> 
#> ── chattr
#> • Supplier: LlamaGPT
#> • Path/URL: [path to compiled program]
#> • Mannequin: [path to model]
#> • Label: GPT4ALL 1.3 (LlamaGPT)

Extending `chattr`

chattr goals to make it straightforward for brand new LLM APIs to be added. chattr
has two parts, the user-interface (Shiny app and
chattr() operate), and the included back-ends (GPT, Copilot, LLamaGPT).
New back-ends don’t have to be added straight in chattr.
If you’re a package deal
developer and want to benefit from the chattr UI, all you want to do is outline ch_submit() technique in your package deal.

The 2 output necessities for ch_submit() are:

As the ultimate return worth, ship the complete response from the mannequin you’re
integrating into chattr.
If streaming (stream is TRUE), output the present output as it’s occurring.
Usually by way of a cat() operate name.

Right here is a straightforward toy instance that reveals easy methods to create a customized technique for
chattr:

library(chattr)
ch_submit.ch_my_llm <- operate(defaults,
                                immediate = NULL,
                                stream = NULL,
                                prompt_build = TRUE,
                                preview = FALSE,
                                ...) {
  # Use `prompt_build` to prepend the immediate
  if(prompt_build) immediate <- paste0("Use the tidyversen", immediate)
  # If `preview` is true, return the ensuing immediate again
  if(preview) return(immediate)
  llm_response <- paste0("You stated this: n", immediate)
  if(stream) {
    cat(">> Streaming:n")
    for(i in seq_len(nchar(llm_response))) {
      # If `stream` is true, make certain to `cat()` the present output
      cat(substr(llm_response, i, i))
      Sys.sleep(0.1)
    }
  }
  # Make certain to return the complete output from the LLM on the finish
  llm_response
}

chattr_defaults("console", supplier = "my llm")
#>
chattr("whats up")
#> >> Streaming:
#> You stated this: 
#> Use the tidyverse
#> whats up
chattr("I can use it proper from RStudio", prompt_build = FALSE)
#> >> Streaming:
#> You stated this: 
#> I can use it proper from RStudio