## AI App Generator [Generate software with AI Create custom business software with a text prompt.](/content/?utm_source=ai-generator-research-lp/index.html) ## Introduction One of the recent projects we've been working on at Glide is an AI app generator. Given a plain-English prompt, it generates a schema, some data, and a basic Glide App. This project started out as a question: how can we make it easier for new users to learn Glide? Sometimes users come to Glide with specific ideas, and other times they just want to experiment. Either way, if they’re they're unfamiliar with some of the nuances and mental models associated with Glide, they may not know how to get started (for example, how to structure their data or how to use advanced features like relations, rollup columns, and user profiles). Ultimately, they may get frustrated trying to make Glide do what they want. ### Basic structure of a Glide App For context, it’s first worth describing the basic structure of a Glide App. In Glide, apps often use data that lives in a data source like Google Sheets or Glide Tables. For example, a grocery-tracking app might use an `Ingredients` table, where each row describes an ingredient. This table might have columns like `name`, `cost`, `color`, etc. Glide can derive complex relationships between the tables, columns, and rows that drive an app. For example, a recipes app might have an associated `Recipes` table, and each item in that table might reference multiple rows in the just-mentioned `Ingredients` table (for example, the recipe for a ham and cheese sandwich might reference the `ham`, `cheese`, and `bread` rows of the `Ingredients` table). In Glide, we call this a relation, and it’s just one type of computation we can run on an app’s underlying data (others include rollups, template strings, and more). A Glide App is intertwined with its data source. For example, changing the data in Google Sheets changes the data that shows up in your app, and vice versa. In fact, Glide uses the schema of your data source (its tables and columns) to generate a default app. If you structure your data in a specific way, Glide can produce relatively intelligent default apps. And, as you work with multiple Glide apps, you’ll build up an intuition for the kinds of schemas that Glide finds meaningful. However, new users won’t have that knowledge or intuition, and this can trip them up. This project’s goal is to give new Glide users a starting point. No matter if they have a specific idea or just want to experiment, we can use the short description they provide to generate a relevant data schema, some data for it, and a basic app they can use to start exploring Glide. And, as they play with the app, they’ll get a better understanding of how to structure their data in ways that unlock some of Glide’s more complex features, like relations. ### GPT-3? We had an inkling that GPT-3 might be a good candidate for doing this given its superb text completion, so we gave it a try. Initially, we started very simple - could GPT-3 output just a relevant schema given a prompt? At first, the answer was no. Over time, though, we found that we could get higher quality results by priming the base, OpenAI models with good, clear examples. There are many nuances to prompt engineering (the process of finding the best prompts to give GPT-3)—your prompts "program the model," and poor or unclear examples yield bad results. In particular, we found that making the most complicated parts of the task simple, clear, and straightforward most significantly improved the quality of our results. As we learned, GPT-3 is essentially like a 3-year-old that’s trained on the entire internet. Slowly, after a couple of examples, we got the hang of it: ### Prompt `description: I want an app that manages orders for my customers``response: You will need a table for storing customer information, a table for order information, and a table for products. Each row in customer information should have a column for name, e-mail address, and phone number. Each row in order information should have a column for id, cost, and product id. Each row in products should have an id and cost.` `description: I want an employee directory that contains information about my employees and where they work.` ### Result `response: You will need a table for storing employee information, a table for storing employee work location, and a table for storing employee phone numbers. Each row in employee information should have a column for name, e-mail address, phone number.` Not bad! But we needed results with more structure. So, we tried this: ### Prompt `Q: I want an app that manages orders for my customers.``A: Tables: Customer Information (with columns for name, e-mail address, phone number), Order Information (with columns for id, cost, list of product ids) ` `Q: I want an employee directory that contains information about my employees and where they work.``A: Tables: Staff (with columns for name, title, phone number, e-mail, location), Locations (with columns for name, address, list of people who work there.` `Q: I want an app that contains information to track issues across different buildings.``A: Tables: Buildings (with columns for name, phone, address, issues), Issues (with columns for subject, notes, building name)` `Q: I want an app to manage my basic expenses.` ### Result `A: Tables: Accounts (with columns for month, amount, category), Expense (with columns for type, amount)` `Q: I want an inventory app to track the items I have in stock.` A: Tables: Inventory (with columns for item_id, date_taken, item name), Items (with columns for item_id, name, quantity)` Better! We could use these results. This work confirmed our first hypothesis: GPT-3 can generate relevant data schemas (tables and columns). But can it generate data, too? The answer here turned out to be a little more unclear. However, for a single-table schema, GPT-3 can _usually_ output a couple of rows of decent data: ### Prompt `Q: Customer Information name | e-mail address | phone number |``A: Jane Smith | jane@gmail.com | 402-784-9003 | John Doe | john@gmail.com | 168-302-4023 | Mark Matthew | mark@gmail.com | 583-234-4391 |``Q: Staff name | title | phone number | e-mail | location |``A: Brandy Fry | Manager | 479-940-2310 | brandy@gmail.com | Menlo Park | Kay Kramer | Software Engineer | 234-284-2039 | kay@gmail.com | Midtown Atlanta | Rebecca Johnson | Product Manager | 129-203-1301 | rebecca@gmail.com | Menlo Park | Locations` `Q: Buildings name | phone | address | issues |``A: Buildings Sunflower | 123-402-3012 | Battery Street | Broken AC, Poor Insulation | Tulip | 302-194-2034 | 767 NW Street | Broken AC |``Q: Order Information id | cost | list of product ids |` ### Result `A: 1 | 100 | {(1, 2, 3)} | 2 | 200 | {(4, 5, 6, 7)} | 3 | 300 | {(8, 9, 10, 11)} |` Another, more promising result: ### Prompt ### Our First Prototype Our experiments weren’t conclusive, but they were strong enough to encourage us to build a prototype: The prototype started out simple: enter a description for an app, and it generates a possible schema. Each title is a table, and each text box is one of its columns. We started playing around, having fun with the tool and its output: ### Detecting data types and generating data Next, we needed to figure out how to detect column data types and generate data (including data that spans across tables). ### Detecting data types Detecting data types turned out to be easy, since we could just add data types to the examples we gave to GPT-3. For example: `Q: I want an app that manages orders for my customers.``A: Tables: Customer Information (with columns for name, e-mail address, phone number), Order Information (with columns for id, cost, list of product ids ), Product (id, cost)` `Q: I want an employee directory that contains information about my employees and where they work.``A: Tables: Staff (with columns for name, title, phone number, e-mail, location), Office Locations (with columns for name, address, list of people who work there` `Q: I want an app that contains information to track issues across different buildings.``A: Tables: Buildings (with columns for name, phone, address, list of issues), Issues (with columns for subject, notes, building name, date )` `Q: I need an app to track my gym workouts.``A: Tables: Workout (with columns for date, duration, list of exercises, weight, notes), Gym Exercises (with columns for name, description, weight`. We could prime GPT with different sets of these inputs, each one specifying a set of tables, columns, and data types. ### Generating Data We decided to generate data for one table at a time. For prompts that produced schemas with multiple tables, we used previous tables as input for the next ones. This helped ensure high table quality (since GPT-3 tends to give worse results for longer input), and it gave GPT-3 the data it needed to generate relations. The entire flow looked something like this: To start, we used the DaVinci model with specific inputs for the schema (orange box), and different inputs for the data (purple boxes). For subsequent tables, in addition to the same set of inputs, we'd feed in the results of the previous tables (as explained above). A couple of days later, we had our first working prototype: ### Making GPT-3 Generation More Stable After we had this basic functionality working inside of Glide, we revisited our GPT-3 experiments. We tried various things, but we eventually settled on doing all of the schema and data generation in a single completion. With this approach, the inputs used to prime GPT-3 looked like this: `Q: I want an app that manages orders for my customers. Tables: 1. Customer Information (with columns for name, e-mail address, phone number), 2. Order Information (with columns for id, cost, list of product ids ), 3. Product (with columns for id, cost)``A: 1. Customer Information name | e-mail address | phone number Jane Smith | jane@gmail.com | 402-784-9003 John Doe | john@gmail.com | 168-302-4023 Mark Matthew | mark@gmail.com | 583-234-4391 Dixie Gale | dixie@gmail.com | 481-231-3349 Natalie Vu | natalie@gmail.com | 921-291-2934 2. Product id | cost 0 | $5 1 | $10 2 | $20 3 | $15 3. Order Information id | cost | list of product ids a | $15 | {0, 1}b | $25 | {0, 2}c | $35 | {2, 3}d | $30 | {1, 2}e | $35 | {0, 1, 2}` ### Final Features Next, we worked on adding some final missing features: - A safety filter to remove harmful or toxic completions. - The ability to open a generated app in the Glide builder, so users can keep customizing it. - Cute defaults to display when GPT-3 crashes (a puppies app, a kitties app, etc.) ### Improving Our GPT-3 Parser We also spent some time tweaking our parser to handle as many GPT-3 failure modes as possible. There was no possible way to catch everything, but we tried to cover as much as possible—missing columns, invalid relations, empty tables, mismatched data, you name it. So, instead, we focused on making the parser as robust as possible. The result was a much more stable parser that covered many, many edge cases, and failed gracefully with a cute default app when things went wrong. ## Lessons learned We learned many things from this project. Here are some key takeaways: - Building a complicated application on top of GPT-3 is surprisingly non-trivial. - Most of the hard work comes down to figuring how to structure your tasks for GPT-3. - If you're building an application that has to parse structured data from a completion, it will be a difficult challenge. - GPT-3 is incredibly smart and incredibly dumb at the same time. - Using AI-augmented software can be amazingly fun. This experiment was both a success and a failure, for many reasons. We learned a lot, and we're excited to keep exploring how AI can make building software with Glide even better!