Skip to content

Playground

The playground is a web app that allows you to interactively build DocETL pipelines. The playground is built with Next.js and TypeScript. We use the docetl Python package (built from this source code) to process the data with a FastAPI server. We stream out the logs from the FastAPI server to the frontend so you can see the pipeline execution progress and outputs in real time.

Why an interactive playground?

Often, unstructured data analysis tasks are fuzzy and require iteration. You might start with a prompt, see the outputs for a sample, then realize you need to tweak the prompt or change the definition of the task you want the LLM to do. Or, you might want to create a complex pipeline that involves multiple operations, but you are unsure of what prompts you want to use for each step, so you want to build your pipeline one operation at a time.

The playground allows you to do just that.

Installation

First, make sure you have installed the DocETL Python package from source. Fork or clone the repository, then run make install in the root directory:

make install

Then, to install the dependencies for the playground, run make install-ui in the root directory.

make install-ui

Then, run make run-ui-dev to start the development server.

make run-ui-dev

Navigate to http://localhost:3000/playground to access the playground.

Setting up the AI Assistant

The UI offers an optional chat-based assistant that can help you iteratively develop your pipeline. It is currently very experimental. It can't write to your pipeline, but you can bounce ideas off of it and get it to help you iteratively develop your pipeline.

To use the assistant, you need to set your OpenAI API key in the .env.local file in the website directory. You can get an API key here. The API key should be in the following format: sk-proj-.... We only support the openai models for the assistant.

Your .env.local file should look like this:

OPENAI_API_KEY=sk-proj-...

Complex Tutorial

See this YouTube video for a more in depth tutorial on how to use the playground.