If you give a monkey a calculator, what do you expect to happen? The monkey picks up the device, inspects it with their hands and probably presses a few buttons in the process. At the end of the inspection there is a result on the screen of the calculator! What does this mean? Can the monkey do mathematical calculations?
The obvious answer to this is no. Punching random numbers into a calculator does not mean the monkey entering those numbers can do math. How did the calculator get the correct answer? Does this mean the calculator understands mathematical concepts?
Again the answer is no. A calculator can do math because it has been programmed to do so, but the calculator does not understand mathematical concepts.
The usefulness of anything returned from a calculator is dependent on the operator inputting the values, and their understanding of the results. A monkey may enter 3 + 5 into a calculator and get 8 as the correct result, but the monkey and the calculator do not understand that 8 is bigger than 3.
On the other hand, a physicist with a calculator will have an easier time figuring out the force required to lift a heavy stone with a pulley. The physicist understands the forces involved and a calculator will make it easier for the physicist to find the answer. Without the understanding of physics, a monkey with a calculator would never be able to reach the same result.
The above example is obvious, and yet here we are with Large Language Models (LLM’s, e.g. ChatGPT, Copilot etc.,). LLM’s are wonderfully impressive at providing knowledge to already understood concepts. Ask an LLM “why is the sky blue?”, or for “a formula to calculate the force required to lift an object with a pulley”, and it will more than likely provide you with a correct answer. Does this mean the LLM understands these concepts, and will replace scientists and physicists?
The answer is less obvious but is still no. As calculators have been programmed to do math, LLM’s have been programmed to answer questions about known concepts. In its simplest form, LLM responses are a probability calculation on what the most likely answer to the question would be. Responses from an LLM are designed to sound like a human and therefore have convincing qualities that give the illusion of understanding. The LLM knows why the sky is blue because scientists have already investigated this topic through experiments and have published papers on their findings. If scientists had not already discovered why the sky is blue, the LLM would not be able to come up with the correct answer.
LLM’s are like a calculator. A calculator can produce the correct answer because the maths that a calculator is capable of are already understood. The calculator does not know that “3 + 5 = 8” or understand anything about the equation. Similarly an LLM does not understand why the sky is blue, but it has been programmed to find the most probable words that may answer the question.
Like a calculator is a tool in a professionals toolkit, so is an LLM. LLM’s can help professionals find information, parse documentation, refine code, summarize text, create images and more, all in the effort to ease a professional's day to day work. In the hands of a professional an LLM is another tool to be used to enhance their work, but the LLM will not replace the knowledge worker.