How to build your own AI Codegen
Code generation from natural language commands is now commonly used in modern development workflows. We'll build a tool that converts natural language instructions into functional Playwright test scripts that can interact with live browser sessions. In this short guide we share a framework for building a natural language to Playwright code generator for any website. Our example uses a NextJS app with Browserbase, but the principles can be adapted for many different programming languages and frameworks.
1. User Interface
Let’s start by building out a simple UI with our installed components for inputting prompts and displaying generated code.
Here are some key features we should consider for the UI:
- Support multi-line input for complex prompts
- Provide clear visual feedback for different states
- User-editable code input to make it possible to extend existing Playwright scripts
2. Connect to a Browserbase page
Before we get to the exciting part of generating Playwright code, we need to pull some data from a running Browserbase session. To do this we’ll create a new file app/actions.ts where we’ll add a NextJS server action to connect to a new Browserbase session and visit the user-inputted URL stored in websiteUrl
3. Code Generation
In the final step we’ll generate Playwright code based on the user’s input prompt and page context pulled from the running Browserbase session. To do this, we need to add a new function to app/actions.ts with the full prompt we send to Claude to generate code.
We must include all the relevant context (current code state, application state, etc) with the prompt to ensure the LLM has all the information it needs. Finally, we should format the response so that we have the output displayed correctly alongside the constraints we’ve put aside.
Let’s update the app/actions.ts file to look like this:
Final Tips
This guide helps you create a codegen app that connects to a Browserbase session, pulls the DOM for a live page, and generates valid Playwright code to perform actions on the page. If you’re building a web agent, this is a great starting point.
This project can be extended to include the Browserbase live view to watch what’s happening on the page, a feedback loop that executes the generated code in the live session, and more tailored prompts for specific use cases. For more advanced use cases, you can explore our open-source AI SDK, Stagehand, which offers additional flexibility and functionality. If you’d like to see these features in action, check out our playground.