A cache that uses Momento as the backing store. See https://gomomento.com.

Hierarchy

Methods

  • Lookup LLM generations in cache by prompt and associated LLM key.

    Parameters

    • prompt: string

      The prompt to lookup.

    • llmKey: string

      The LLM key to lookup.

    Returns Promise<null | Generation[]>

    The generations associated with the prompt and LLM key, or null if not found.

  • Update the cache with the given generations.

    Note this overwrites any existing generations for the given prompt and LLM key.

    Parameters

    • prompt: string

      The prompt to update.

    • llmKey: string

      The LLM key to update.

    • value: Generation[]

      The generations to store.

    Returns Promise<void>

  • Create a new standard cache backed by Momento.

    Parameters

    Returns Promise<MomentoCache>

    The Momento-backed cache.

    Throws

    InvalidArgumentError if props.ttlSeconds is not strictly positive.

Generated using TypeDoc