Stephen Malina

This is my blog. There are many others like it but this one is mine.

GPT4 Predictions

As a way to keep myself honest, I’m going to be recording GPT4 predictions here and archiving versions of the post so I can’t go back and change it without people knowing. Not that I would do this anyway, but it’s been a tough week for “trust me, I’m telling the truth” so I’m doing this in case readers want the receipts.

I predict GPT4 won’t be able to…

  • Write a blog post about gene therapy that I consider to be of equivalent quality to my own
  • Write a scientific paper about machine learning. As many prompts as possible allowed but the entire paper’s text needs to be written by the model with no editing.
  • Solve three problems from a recent programming competition on Leetcode (Manifold market)
  • Describe the analogy between different framings of machine learning concepts
    • E.g. describe the Bayesian perspective on training deep learning models
  • Correctly solve 10 randomly chosen problems from Linear Algebra Done Right without chain-of-thought prompting or few-shot examples.
  • Describe a coherent design for a machine learning model training platform and answer my questions about it
  • Find the solution to an unsolved as of its deployment scientific problem.
  • Play Nethack better than the winner of 2021’s NeurIPS Nethack challenge.

I predict GPT4 will be able to…

  • Solve >18/20 multiplication problems I come up with that have between 3-6 digits in the multiplicands.
  • Write internet marketing-style blog posts about relatively popular topics
    • Ex: “You’re marketing a new product that tracks my keyboard presses to determine where I’m spending my time. The following is copy for a press release about it.”

Appendix

GPT3’s current attempts at some of these tasks

Requires GPT3 API access.

Archived versions

comments powered by Disqus