I’ve been spending some time thinking about this issue. I even emailed Harry Binswanger in February regarding it, sharing some of my ideas on how to apply Objectivist epistemology to AI. However, my views have shifted somewhat since then. Here is what I currently think:

1. Most of the current mainstream approaches to AI are suboptimal; although they clearly provide value, they are limited by what I argue is the wrong architecture.
2. AI is fundamentally about finding the most efficient computer program for a specific purpose you want to achieve.

Most of the current AI approaches are heavily limited by what I’d describe as a “fixed” architecture: the AI program always runs the same set of instructions with just different parameters (and the training phase consists of finding the best parameters). LLMs can somewhat work around this by virtue of being able to output code, but they’re really inefficient and struggle with coding anything non-trivial. The most glaring issue with this “fixed” architecture limitation is the fact that most deep-learning approaches require massive amounts of computation to achieve their desired purpose.

So, what do I think is the right approach? Finding the algorithm through which the computer can receive English (or another natural language) as input and then output a valid and efficient program to achieve the desired purpose, either through machine code or some intermediate representation. This program might run on either the CPU or the GPU! This is where I think Objectivist epistemology might be able to help, though I have yet to figure out the exact details of how to proceed. I think it would involve something analogous to concept formation, with the main purpose of understanding English inputs and the hardware itself. There are a few questions I’ve been pondering here, and I might be able to collaborate with James if he’s interested.

One might wonder how this would handle certain tasks, like asking an LLM a question about history when the user doesn’t explicitly intend to generate a program. I think this would likely be handled by something like a “history facts” program generated by my proposed AI model; this program could either preprocess a vast set of history facts or browse the internet for information. To provide a general-purpose AI experience like LLM chatbots, I imagine it would be a single program that either chooses an existing program (if there is one fit for the task) or generates a new one if necessary.

In summary, my main points are:

1. AI should have a strong “understanding” of code and English text.
2. AI should be able to generate code that is most appropriate for the hardware; modern CPUs and GPUs are often limited by memory bandwidth rather than sheer math operations.
3. Using a single general-purpose model is inefficient as it lacks context. Running the same program in the same way for both chess and history facts will lead to wasted computation.

/sb