Arxiv Preprint – Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation

In this episode we discuss Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
by Eric Zelikman, Eliana Lorch, Lester Mackey, Adam Tauman Kalai. The paper discusses how a language-model-infused scaffolding program uses a seed “improver” program to iteratively improve itself by querying a language model multiple times and optimizing based on a utility function. The improved improver, after self-enhancement, outperforms the original and applies advanced strategies like beam search, genetic algorithms, and simulated annealing, though not achieving true recursive self-improvement because the underlying language models remain unchanged. The study utilized GPT-4 to demonstrate self-improvement capabilities and addressed concerns about the potential of self-improving technology, including the evaluation of sandbox security bypasses by the generated code.