jest integration

Baserun offers a jest preset, which will automatically add the beforeAll to your test file and turn all it and test calls into baserun.trace calls.

Setup

npm install baserun
# or
yarn add baserun

There is no particular change needed in your actual tests, just add the --preset baserun flag to your jest command.

test_module.spec.ts

import { baserun } from "baserun";

describe("My Test Suite", () => {
  // ... the tests
});

npm run jest --preset baserun test_module.spec.ts

Running multiple tests

It’s often helpful to exercise the same test over multiple examples. To do this, we suggest doing a simple for loop to autogenerate your tests and for larger numbers of examples you can read from a file or other data structure.

my-test.test.ts

import { baserun } from "baserun";

const examples = [
  { place: "Paris", expected: "Eiffel Tower" },
  { place: "Rome", expected: "Colosseum" },
];

describe("Baserun end-to-end", () => {
  beforeAll(async () => {
    // the only thing needed to make baserun work
    await baserun.init();
  });

  for (const { place, expected } of examples) {
    it(`should suggest the ${expected}`, async () => {
      const chatCompletion = await openai.chat.completions.create({
        model: "gpt-3.5-turbo",
        temperature: 0.7,
        messages: [
          {
            role: "user",
            content: `What are three activities to do in ${place}?`,
          },
        ],
      });

      expect(chatCompletion.choices[0].message!.content!).toContain(expected);
    });
  }
});

Existing presets

If you are already using a Jest preset such as ts-jest you will need to merge the presets in a Jest config

// jest.config.js or jest.config.baserun.js

const tsPreset = require("ts-jest/jest-preset");
const baserunPreset = require("baserun/jest-preset");

module.exports = {
  ...tsPreset,
  ...baserunPreset,
  testTimeout: 10000,
};

Naming

Baserun lets you compare test results and debug each step side by side. We consider the full path of the describe and it names of the jest test as a unique identifier for the test. In the example above, two tests would be run and identified by ("Baserun end-to-end", "should suggest the Eiffel Tower") and ("Baserun end-to-end", "should suggest the Colosseum")

Automatic evaluations

Evaluate tests’ outputs.

Introduction

Prompt playground

Python SDK 2.0

Get started with SDK

Monitoring

Prompt templates

Evaluation

Testing

Datasets

Fine-tune

Integrations

jest integration

Setup

Running multiple tests

Existing presets

Naming

Automatic evaluations

​Setup

​Running multiple tests

​Existing presets

​Naming

Automatic evaluations

Setup

Running multiple tests

Existing presets

Naming