r/AskComputerScience • u/kater543 • 13d ago
How efficient are 1st, 2nd, 3rd, and 4th+ generation languages?
1st gen Binary machine code in my knowledge is “100%”(is it really? Not knowledgeable enough) efficient in theory given it directly converses with the machine. How much overhead does each generation of compilers and other inefficiencies add? Like how much overhead does like assembly add? C? C++? C#? Python?
I’ve always heard more human language like languages are less efficient. If that’s the case why don’t we use LLMs to convert/write lower level languages like machine code or assembly?
11
u/teraflop 13d ago
This question doesn't really make sense and isn't really answerable, for multiple reasons.
First of all, the whole idea of language "generations" is not really rigorous at all and has mostly been forgotten. It's basically just a marketing term: when compiled languages came around, somebody had the idea of labeling "machine code", "assembly code" and "compiled languages" as "1GL/2GL/3GL".
And after that "4GL" got used as a buzzword for any number of languages that had very little in common, except for having extra features or unconventional designs compared to the predecessors that were in common use at the time. So it isn't sensible to ask whether 4GL languages as a whole are "faster" or "slower" than 3GL, because you're just lumping too many completely unrelated technologies together.
For more modern languages, such as JavaScript or Python or Rust, nobody even bothers trying to classify them into "generations". It's just too vague a concept to be useful.
But suppose we just limit ourselves to asking whether "compiled languages" are faster than "machine code languages". This still doesn't make sense, because "compiled languages" are generally compiled to machine code!
So what you're really asking is mainly a question about the implementation (the compiler), not the language itself. That is, does the machine code generated by a particular compiler, in a particular situation, perform better than machine code generated by some other method?
For instance, the C language hasn't changed dramatically in the last 40 years or so, and yet C compilers nowadays are much more sophisticated and better at optimization than the compilers of 40 years ago. So once again, it makes no sense to talk in generalities when the answer depends on the details of a specific compiler.
As for the question of whether it makes sense for LLMs to generate machine code directly: at this point I see absolutely no reason to think that's a sensible approach.
Consider the problem of sorting a long list of integers. It makes no sense to try to solve this problem by feeding all the numbers to an LLM. We have existing deterministic sorting algorithms that require vastly less computation and are vastly more reliable.
In a similar way, a compiler can translate from the syntax of a high-level language to machine code in a mechanical way that we know to be reliable (at least assuming the compiler doesn't have bugs).
To the extent that the compiled program's performance depends on the compiler's choices, these choices are usually much more amenable to standard search algorithms (e.g. graph-coloring register allocation) than just asking an LLM to take its best guess at the correct choice.
If there's a future for LLMs in compilation, it almost certainly makes much more sense to have the LLM guide the compiler, e.g. by identifying patterns and proposing ways to transform the code and make it more efficient. For instance, maybe the LLM could spot a case where nested loops could be unrolled or interchanged to provide better cache locality, based on its knowledge of the program's overall structure.
But it almost certainly only makes sense to apply this much energy-intensive computation to the "hot" inner loops where performance actually matters. In a typical program, the vast majority of the code is non-performance-critical.
4
u/apnorton 13d ago
I'm not going to write up a long answer, but you might be interested in a segment of this interview by Bjarne Stroustrup, in which he discusses the cost of abstractions and how C++ has, in his word, "negative overhead abstraction:" https://youtu.be/U46fJ2bJ-co?t=4918
Compilers tend to be far better at writing assembly than humans are, and so it can actually be far more efficient to write abstractions in C++ than to attempt to implement the notion by hand in assembly. The same is true for some other languages, as well, but C++ is the easiest example that came to mind.
1
u/MistakeIndividual690 13d ago
Templating in particular in C++ can give it the edge over C— yes it’s possible to manually (or with macros) write a separate version of a linked list for every type you have in C, but people generally don’t want to do that—it’s messy and creates more maintenance headaches. In C++ that can be automatic, which generates more, but likely faster code.
1
2
u/nuclear_splines Ph.D Data Science 13d ago
In principle a compiler adds no inefficiency: it translates a higher level language into machine code instructions, the same as you might have written by hand in assembly. We don't use LLMs to perform that translation, because that's what compilers already do, better. A skilled assembly programmer may be able to write more efficient machine instructions than a compiler in some circumstances, but this is getting pretty far into the weeds.
By contrast, an interpreted language like Python does incur an efficiency hit. When we write:
def foo(a, b):
print(a + b)
What are a and b? Integers? Floats? Strings? Does a + b perform arithmetic addition, or string concatenation? If integers, do they need to be translated into a string before printing? We cannot know ahead of time how much memory this function uses or what instructions it's going to run -- these facts must be determined at runtime by checking the types of the variables and figuring out the appropriate operation to call for +.
It's not really possible to translate the above Python function to a lower-level language like C or assembly without understanding the intent of the program. If we know that function is meant to take two integers and print their sum, then we can write an equivalent assembly program that's much faster, but this loses the flexibility of an interpreted language. Without that knowledge, the best we could do is write machine code that performs the same checks -- see whether a and b are ints, floats, or strings, determine whether any typecasts are necessary and whether we're calling addition or string concatenation, and so on. At this point, you haven't really made a performance gain over the interpreted language.
There's a lot more nuance and I've cherry-picked an illustrative example, but I think that's a starting point for understanding the "efficiency and overhead" of languages you're getting at.
1
u/mxldevs 13d ago
Does this mean by simply specifying type hints that were introduced to Python, it is already going to be much more efficient than not having it?
5
u/lgastako 13d ago
Unfortunately the type hints in python do nothing at runtime, so they don't affect performance at all.
1
u/MistakeIndividual690 13d ago
In some languages adding typing does improve performance— many functional languages for example. In Python, currently, unfortunately, no.
Although I would expect Python to migrate this way in the future.
1
2
u/CS_70 12d ago
"Machine code" has not been "machine code" for a while, in many cpus more complex instructions are implemented as sequences of simpler commands.
Assuming you talk of optimization in time (aka speed) and not in space or cost, in general a compiler does not add overhead, on the contrary: it's much safer to let the compiler optimize the higher level construct than leave it to a programmer. The average programmer knows far less tricks, even when programmers knew how to program in assembly.
It is (runtime) language features that may add overhead: for example a high level language that has runtime type check will always do more than a high level language which does not.
Also of course using libraries - a library is by definition a generic solution to a generic problem, so occasionally chances are you could re-implement a slightly faster solution specific to your problem by hand, but the cost tradeoff seldom is worth it.
"Generations" is a very loose terms indicating different things and trading imperative semantics with more human-like types.
1
u/recursion_is_love 12d ago
The concept of the term efficiency is too vague. You would need a measurement that more quantifiable.
The cost of abstraction in term of additional instruction need is not always high, there are what is called zero-cost abstraction in many cases.
1
u/mplang 12d ago
Some great answers here already, so I'll just chime-in to add that the idea of vibe-coding assembly is genuinely terrifying, and in some respects the most inefficient way to generate code.
LLMs aren't some magic, perfect code generating machines. A lot of the illusion that they are is enabled by the expressiveness of languages like Python, and we're generating disposable software where durability and correctness matters less than upvotes and quarterly earnings.
1
u/Traveling-Techie 12d ago
Efficiency isn’t the only goal. If you’re using an active web page the network latency is usually much larger than the execution time. Maintainability is another important goal.
1
u/NumberInfinite2068 8d ago
None.
There is no reason you cannot compile a 3GL or 4GL to a 100% optimal binary, or at least as optimal as the equivalent assembly or machine code.
The "more human" stuff makes no difference.
1
u/Such-Perception4154 8d ago
There is also a question of "how hard would it be to write peak efficiency code for the given language", and it's not gonna be easy for assembly for sure. Especially for large projects.
1
u/lizardhistorian 8d ago edited 8d ago
As a gross generalization:
1st: Presumed asymptotically optimal
2nd: 400% overhead
3rd: 2,000% overhead
4th: 20,000% overhead
All of the higher-level languages optimize for the "hot paths" of the common things that need to be done by short-cutting back to a lower-level implementation for that path. So those things will elide the intrinsic overhead so programs written in the higher-level languages do not run this slow.
A great example would be the native Python asyncio vs. the short-cut uvloop. This alone can yield a 200% ~ 400% improvement from just this one hotpath.
Those questioning the overhead on the 2nd generation the example would be a lack of operators in the language that the processor provides notably -with-carry operations such as shift-with-carry. If there were no compiler optimization that specifically detect these cases it would turn one instruction into a loop.
1
u/No_Score_1977 8d ago
In terms of compiled languages, the 'higher' level it is, the better chance you have of compiling it more efficiently.
The more semantic information a compiler has, then it has more to play with when doing the compilation.
For example, if your language has a 'number' type, but no more information than that, that it cannot ever be as efficient as a language that has say a 'byte' or 'int8' type, as with less type information, you simply cannot safely use the smaller type.
However, this is all theoretical, the compiler has to actually do all this stuff (in a simple case like this, they will).
The more information you give a compiler, the better job it can do.
To get more efficient code, the language needs to be further from the metal, that allows the compiler to get closer.
15
u/RICoder72 13d ago
Its not quite so simple as efficiency in code is often dictated by what you are trying to be efficient for. In other words, what are you optimizing for? So there are a few answers.
Lets assume that all code written is 100% as efficient as possible for that language (so 100% is baseline not necessarily a pure perfect efficiency). In assembly, it is fair to say that you are at the 100% mark. As you progress to 2nd and 3rd Gen languages you are passing through a compiler. That compiler is actually breaking down your code (say C++) into assembly as well as it can. These compilers are REALLY good at generating efficient assembly, but not perfect. Maybe they get a 95 to 98% mark at best. There are actually commands in many of these languages to write inline assembly (like a PRAGMA) specifically for the case where you dont trust the compiler to get it right.
Now you get into interpreted languages and there are two flavors (someone is going to fight me on this). One is code running in an interpreter, the other is a flavor of JIT. We will start with the JIT.
For languages like C# or Java (and Java used to be purely interpreted) you get this middle ground where a compiler compiles your code down to byte-code or something symbolic. It isn't written for the specific OS or chip, it is written for a Just In Time compiler to read and compile on the fly. So you write some portable code in C#, put it on some machine as an executable file, and the machine checks if it is actually compiled or in some symbolic state. If it isn't compiled, it does it right then (just in time) and we are back where we were with the previous generation like C++. Theoretically just as efficient.
Finally we are at the interpreted languages. Python, Javascript, whatever. These languages dont compiler down to a lower level language. They get processed through an interpreter which is just another program reading your program and telling the computer what to do. You could write perfect code in those languages and you are still always going to pay a penalty in efficiency because it has no choice but to pass through a software layer rather because executing commands on the chip.
So, I think the answer is that its either compiled or it isn't. If it is, you are capable of writing nearly 100% efficient code. If it isnt, you are going to pay an efficiency penalty.