Function Observers

Examples

Observing model calls or tool executions can be useful for many reasons. You can for example use them for logging, to send functional call data to observability APIs, or to record function calls in your DB.

ModelFusion provides a simple way to observe function calls. Function observers are called when function events occur. The supported functions are model calls and tool executions.

Function Events

Function events have several standard properties, as well as event-specific properties. The standard properties are:

• timestamp: The timestamp of the event.
• eventType: The type of the event. Can be started or finished.
• functionType: The type of the function, for example, text-generation.
• callId: A unique ID for the function call.
• parentCallId: The ID of the parent function call. Optional.
• functionId: A unique ID for the function. Optional.
• runId: A unique ID for the run. Available when the function is called with a run.
• sessionId: A unique ID for the session. Available when the function is called with a run that has a session.
• userId: A unique ID for the user. Available when the function is called with a run that has a user.

Events with eventType = 'started' also have a startTimestamp property.

Events with eventType = 'finished' have the following additional properties:

• startTimestamp: The timestamp of the function start.
• finishTimestamp: The timestamp of function execution completion.
• durationInMs: The duration of the function execution in milliseconds.
• result: The result of the function execution.
  • status: Can be success, erroror abort.
    • success: The function execution was successful. Additional information such as the original response, the output, and usage information are available depending on the function type and model.
    • error: The function execution failed. The error is available in the error property.
    • abort: The function execution was aborted.

In addition to these properties, specific events contain e.g. the settings or outputs that are relevant for their function type. You can access them in a type-safe manner by selecting (e.g. via if or switch) on the functionType, eventType and result.status properties.

Usage

FunctionObserver

API

FunctionObserver has a single method, onFunctionEvent that is called with a FunctionEvent when it occurs. You can implement it to do whatever you want with the event.

Example: Custom function observer

const observer: FunctionObserver = {
  onFunctionEvent(event: FunctionEvent) {
    // you could also switch on e.g. event.functionType
    switch (event.eventType) {
      case "started": {
        console.log(
          `[${event.timestamp.toISOString()}] ${event.callId} - ${
            event.functionType
          } ${event.eventType}`
        );
        break;
      }
      case "finished": {
        console.log(
          `[${event.timestamp.toISOString()}] ${event.callId} - ${
            event.functionType
          } ${event.eventType} in ${event.durationInMs}ms`
        );
        break;
      }
    }
  },
};

Global function observers

You can set global function observers that are called for all function events.

Example

import { generateText, openai, modelfusion } from "modelfusion";

// Set a global function observer:
modelfusion.setFunctionObservers([customObserver]);

const text = await generateText({
  model: openai.CompletionTextGenerator({
    model: "gpt-3.5-turbo-instruct",
    maxGenerationTokens: 50,
  }),
  prompt: "Write a short story about a robot name Evo:\n\n",
});

Function-scoped function observers

You can set function observers for specific functions in the function options.

Example

const text = await generateText({
  model: openai.CompletionTextGenerator({
    model: "gpt-3.5-turbo-instruct",
    maxGenerationTokens: 50,
  }),
  prompt: "Write a short story about a robot named Nox:\n\n",
  observers: [observer],
});

Run-scoped function observers

You can set function observers for all functions calls with a run.

Example

const run = new DefaultRun({
  observers: [observer],
});

const text = await generateText({
  model: openai.CompletionTextGenerator({
    model: "gpt-3.5-turbo-instruct",
    maxGenerationTokens: 50,
  }),
  prompt: "Write a short story about a robot named Pam:\n\n",
  run,
});

Model-scoped function observers

You can set function observers for all functions calls that use a model instance.

Example

const text = await generateText({
  model: openai.CompletionTextGenerator({
    model: "gpt-3.5-turbo-instruct",
    maxGenerationTokens: 50,
    observers: [observer],
  }),
  prompt: "Write a short story about a robot name Bud:\n\n",
});