[LINUX] nl command processing performance

The processing performance of nl commands and commands with similar functions is summarized.

Processing speed ranking

The time required to add the line number of 1000000 lines of each implementation is summarized. The following alternative methods are also included in the comparison.

-Alternate nl command with various commands --Alternate nl command with script --Alternate nl command in editor

The GNU implementation occupies the top position. In GNU, cat -n and grep -n are faster than nl.

time(real) command version
0.372 ggrep ggrep (GNU grep) 3.6
0.376 gcat cat (GNU coreutils) 8.32
0.401 gnl nl (GNU coreutils) 8.32
0.610 mawk MAWK Version 1.3.4
0.676 nl BSD
0.780 grep grep (BSD grep) 2.5.1-FreeBSD
0.786 busybox nl(* Converted value) BusyBox v1.30.1 (Ubuntu 1:1.30.1-4ubuntu6.3) multi-call binary.
1.021 cat BSD
1.106 gawk GNU Awk 5.1.0, API: 3.0 (GNU MPFR 4.1.0, GNU MP 6.2.1)
1.979 sed BSD
1.996 gsed gsed (GNU sed) 4.8
2.428 nawk awk version 20070501 - one true awk(new awk)
2.751 perl perl 5, version 32, subversion 0 (v5.32.0) built for darwin-thread-multi-2level
5.558 vim VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Nov 29 2020 16:14:01)

Implementation comparison

The measurement environment is as follows.

Basically, it is measured on macOS, and only busybox is measured and converted on Ubuntu. (Busybox relies on Linux header files, which was annoying)

nl implementation comparison

coreutils vs BSD macOS 10.13.6 / Intel(R) Core(TM) i7-2677M CPU @ 1.80GHz (2C4T)

$ time seq 1000000 | nl >/dev/null
time 1 2 3 Ave.
real 0m0.678s 0m0.675s 0m0.676s 0.676
user 0m1.072s 0m1.065s 0m1.065s 1.067
sys 0m0.019s 0m0.020s 0m0.018s 0.019
$ time seq 1000000 | gnl >/dev/null
time 1 2 3 Ave.
real 0m0.401s 0m0.404s 0m0.399s 0.401
user 0m0.760s 0m0.770s 0m0.758s 0.763
sys 0m0.015s 0m0.015s 0m0.014s 0.015

coreutils vs busybox Ubuntu 20.04LTS / Intel(R) Core(TM) i5-2415M CPU @ 2.30GHz (2C4T)

$ time seq 1000000 | nl >/dev/null							
time 1 2 3 Ave.
real 0m0.184s 0m0.183s 0m0.185s 0.184
user 0m0.188s 0m0.191s 0m0.189s 0.189
sys 0m0.018s 0m0.014s 0m0.018s 0.017
$ time seq 1000000 | busybox nl >/dev/null							
time 1 2 3 Ave.
real 0m0.364s 0m0.363s 0m0.355s 0.361
user 0m0.376s 0m0.397s 0m0.378s 0.384
sys 0m0.029s 0m0.005s 0m0.005s 0.013

Comparison with other commands

This command was used in Substitute nl command with various commands. I haven't tried less. macOS 10.13.6 / Intel(R) Core(TM) i7-2677M CPU @ 1.80GHz (2C4T)

cat -n (coreutils vs BSD)

$ time seq 1000000 | gcat -n >/dev/null
time 1 2 3 Ave.
real 0m0.377s 0m0.373s 0m0.379s 0.376
user 0m0.410s 0m0.407s 0m0.413s 0.410
sys 0m0.017s 0m0.017s 0m0.017s 0.017
$ time seq 1000000 | cat -n >/dev/null
time 1 2 3 Ave.
real 0m1.026s 0m1.012s 0m1.026s 1.021
user 0m1.417s 0m1.396s 0m1.414s 1.409
sys 0m0.020s 0m0.020s 0m0.020s 0.020

grep -n (GNU vs BSD)

$ time seq 1000000 | ggrep -n ^ >/dev/null
time 1 2 3 Ave.
real 0m0.368s 0m0.370s 0m0.378s 0.372
user 0m0.365s 0m0.367s 0m0.374s 0.369
sys 0m0.015s 0m0.015s 0m0.016s 0.015
$ time seq 1000000 | grep -n ^ >/dev/null
time 1 2 3 Ave.
real 0m0.786s 0m0.774s 0m0.780s 0.780
user 0m1.160s 0m1.149s 0m1.167s 1.159
sys 0m0.019s 0m0.020s 0m0.020s 0.020

Comparison with script

This is the script one-liner used in Substitute nl command with script. macOS 10.13.6 / Intel(R) Core(TM) i7-2677M CPU @ 1.80GHz (2C4T)

Awk(gawk vs mawk vs nawk)

gawk


$ time seq 1000000 | gawk '$0=NR"\t"$0' >/dev/null
time 1 2 3 Ave.
real 0m1.103s 0m1.126s 0m1.089s 1.106
user 0m1.464s 0m1.528s 0m1.476s 1.489
sys 0m0.023s 0m0.022s 0m0.020s 0.022

mawk


$ time seq 1000000 | mawk '$0=NR"\t"$0' >/dev/null
time 1 2 3 Ave.
real 0m0.609s 0m0.613s 0m0.609s 0.610
user 0m1.000s 0m1.006s 0m1.005s 1.004
sys 0m0.021s 0m0.021s 0m0.021s 0.021

nawk


$ time seq 1000000 | /usr/bin/awk '$0=NR"\t"$0' >/dev/null
time 1 2 3 Ave.
real 0m2.444s 0m2.460s 0m2.379s 2.428
user 0m2.845s 0m2.866s 0m2.790s 2.834
sys 0m0.039s 0m0.033s 0m0.036s 0.036

Perl

$ time seq 1000000 | perl -pe 's/^/$.\t/' >/dev/null
time 1 2 3 Ave.
real 0m2.709s 0m2.776s 0m2.768s 2.751
user 0m3.078s 0m3.160s 0m3.147s 3.128
sys 0m0.035s 0m0.030s 0m0.030s 0.032

Comparison with editor

This is the editor used in Substitute nl command with editor. macOS 10.13.6 / Intel(R) Core(TM) i7-2677M CPU @ 1.80GHz (2C4T)

SED(GNU vs BSD)

$ time seq 1000000 | gsed = | paste - - >/dev/null
time 1 2 3 Ave.
real 0m1.994s 0m1.996s 0m1.997s 1.996
user 0m3.047s 0m3.069s 0m3.052s 3.056
sys 0m0.033s 0m0.035s 0m0.037s 0.035
$ time seq 1000000 | sed = | paste - - >/dev/null
time 1 2 3 Ave.
real 0m1.974s 0m1.991s 0m1.973s 1.979
user 0m3.014s 0m3.047s 0m3.026s 3.029
sys 0m0.043s 0m0.043s 0m0.044s 0.043

Strictly speaking, it also depends on the implementation of paste.

Vim

$ time seq 1000000 | vim -es /dev/stdin +':set number' +%p +q! >/dev/null
time 1 2 3 Ave.
real 0m5.580s 0m5.551s 0m5.543s 5.558
user 0m4.918s 0m4.916s 0m4.928s 4.921
sys 0m0.778s 0m0.743s 0m0.725s 0.749

Recommended Posts

nl command processing performance
Participate in nl command development
Command line argument processing (Python docopt)
Summary of "nl command Advent Calendar 2020"