GPT4 Predictions
As a way to keep myself honest, I’m going to be recording GPT4 predictions here and archiving versions of the post so I can’t go back and change it without people knowing. Not that I would do this anyway, but it’s been a tough week for “trust me, I’m telling the truth” so I’m doing this in case readers want the receipts.
I predict GPT4 won’t be able to… #
- Write a blog post about gene therapy that I consider to be of equivalent quality to my own
- Write a scientific paper about machine learning. As many prompts as possible allowed but the entire paper’s text needs to be written by the model with no editing.
- Solve three problems from a recent programming competition on Leetcode (Manifold market)
- Describe the analogy between different framings of machine learning concepts
- E.g. describe the Bayesian perspective on training deep learning models
- Correctly solve 10 randomly chosen problems from Linear Algebra Done Right without chain-of-thought prompting or few-shot examples.
- Describe a coherent design for a machine learning model training platform and answer my questions about it
- Find the solution to an unsolved as of its deployment scientific problem.
- Play Nethack better than the winner of 2021’s NeurIPS Nethack challenge.
I predict GPT4 will be able to… #
- Solve >18/20 multiplication problems I come up with that have between 3-6 digits in the multiplicands.
- Write internet marketing-style blog posts about relatively popular topics
- Ex: “You’re marketing a new product that tracks my keyboard presses to determine where I’m spending my time. The following is copy for a press release about it.”
Appendix #
GPT3’s current attempts at some of these tasks #
Requires GPT3 API access.
- Explaining deep learning from a Bayesian perspective
- Writing a paper about protein scaling laws
- Writing a design doc for an extension to a machine learning platform