x

Programs, configuration and documentation that don't fit anywhere else
Log | Files | Refs | README | LICENSE

llm.1 (2042B)

      1 .Dd
      2 .Dt LLAMA 1
      3 .Sh NAME
      4 .Nm llm
      5 .Nd chat with a remote large language model
      6 .Sh SYNOPSIS
      7 .Nm
      8 .Op Fl c
      9 .Op Fl m Ar model
     10 .Op Fl s Ar prompt
     11 .Sh DESCRIPTION
     12 .Nm
     13 starts a chat with a large language model.
     14 The prompt is read from the standard input
     15 and the reply is written to the standard output.
     16 Any model available through
     17 the OpenAI-compatible chat completion HTTP API
     18 can be used.
     19 .Pp
     20 A back-and-forth chat may be started using the
     21 .Fl c
     22 flag.
     23 In this mode,
     24 a line consisting of just a literal dot character
     25 .Pq "."
     26 sends the prompt.
     27 Subsequent replies and prompts are included as context for the model's responses.
     28 .Pp
     29 A configuration file written to
     30 .Pa $HOME/.config/openai
     31 will direct
     32 .Nm
     33 how to connect to the chat completion HTTP API
     34 and set any completion options.
     35 The file consists of key-value pairs separated by whitespace,
     36 one per line.
     37 Blank lines and lines beginning with "#" are ignored.
     38 The following options may be set:
     39 .Bl -tag -width Ds
     40 .It Ic url Ar url
     41 The base URL where an OpenAI-compatible HTTP API is served.
     42 The default is
     43 .Ar http://127.0.0.1:8080 .
     44 .It Ic token Ar string
     45 The bearer token used to authenticate requests against the HTTP API.
     46 .It Ic model Ar name
     47 Request prompts be completed by model
     48 .Ar name .
     49 .El
     50 .Pp
     51 The following command-line flags are understood:
     52 .Bl -tag -width Ds
     53 .It Fl c
     54 Start a back-and-forth chat.
     55 .It Fl m Ar model
     56 Prompt
     57 .Ar model .
     58 Note that
     59 .Xr llama-server 1
     60 from llama.cpp ignores this value.
     61 .It Fl s Ar prompt
     62 Set
     63 .Ar prompt
     64 as the system prompt.
     65 .El
     66 .Sh EXAMPLES
     67 .Pp
     68 Chat with a locally-hosted Mistral NeMo model:
     69 .Bd -literal -offset Ds
     70 llama-server -m models/Mistral-Nemo-Instruct-2407-Q6_K.gguf -c 16384 -fa &
     71 echo "Hello, world!" | llm
     72 .Ed
     73 .Pp
     74 A configuration file to use Mistral's hosted language models,
     75 using a small model if none is specified at the command line:
     76 .Bd -literal -offset Ds
     77 url https://api.mistral.ai
     78 token abcdef12345678
     79 # use smaller models to boil oceans a bit more slowly
     80 model ministral-8b-latest
     81 # model mistral-small-latest
     82 .Ed
     83 .Sh EXIT STATUS
     84 .Ex