Someone said flat out in a meeting that awk is faster than C, the way you state a fact everyone already knows, and half the room nodded even though the claim is wrong and the instinct behind it is still right.
I let it go in the moment, then spent too long working out why people who know what they are doing keep repeating a false claim they have watched come true over and over.
Hand someone a few gigs of log and a question, and the awk person has an answer before the Python script has finished importing its libraries.
mawk starts in about nothing, and someone wrote its inner loop, the part that splits a line into fields and runs a rule against them, in tight C years ago. When the job is exactly that shape, you're running C without having typed any, and most of these jobs are waiting on the disk anyway. Everything finishes at about the same wall-clock time, so the tool that's quickest to type feels fastest.
Faster than C is wrong, and it isn't close, because awk is a C program and a decent C program can match anything it does. The comparison people are making is with the C they were ever actually going to write, which awk can beat on a lot of real work.
Line-shaped work
Counting, summing, filtering, reshaping fields on lines of text. One line gives you the answer, and the disk is the real bottleneck anyway.
awk wins on time-to-answer
Everything else
Random access, real data structures, number-crunching, anything processor-bound. NumPy and a real C program pull ahead, and the startup edge stops mattering.
C is the ceiling
Give it random access or a real data structure and now you're fighting the language instead of using it. NumPy laps it on numbers, a real C program laps it on anything tight, and the startup edge you loved vanishes the second the job runs ten minutes instead of fifty milliseconds.
I don't correct this unless the distinction matters, because people usually mean awk got the answer before anybody wrote the C program, and that is true.
Where this stands
I reach for awk first on small, disk-bound jobs over lines of text and stop when I need real data structures or the processor becomes the bottleneck.
- A careful benchmark showing mawk beating a competently written, buffered C program on a task that's processor-bound rather than disk-bound.
- Evidence that awk's startup and inner-loop advantages hold once the work stops being record-oriented field processing.