Command line tricks
30 May 2024
I'm doing a little talk at my department's developer group on modern command line tricks, so though I would mirror that talk with a blog post about new terminal tools and speedy tips.
To avoid tools that pollute my environment too much I'm running all of these in a fresh Alpine Linux virtual machine.
montblanc:~/vd_agent$ uname -a
Linux montblanc 6.6.32-0-lts #1-Alpine SMP PREEMPT_DYNAMIC Fri, 24 May 2024 10:11:26 +0000 x86_64 Linux
I've been researching new tools and had a list I wanted to try out. So let's start at the top.
Shell zero overheads
Some things that you might not know that are very likely already on your machine and in your shell.
-
Ctrl-r
Backwards search through your history.
-
cd -
,pushd
,popd
anddirs
Used to create a stack of directories. Useful for saving where you are and quickly coming back to it. For example, say I was in a certain directory and needed to get back to it after an accidental
cd
, thencd -
will take me back to the last directory I was in:$ pwd /home/titus/.config/broot $ cd && pwd /home/titus $ cd - /home/titus/.config/broot
The commands
pushd
andpopd
generalize this to a directory stack. You can view the stack at any time withdirs
. Let's see how that works:$ pwd /home/titus/.config/broot # use pushd to add a specific else current directory to the stack $ pushd ~/entr/ ~/entr ~/.config/broot # cd around a bit $ cd # check the directory stack $ dirs ~ ~/.config/broot # pop to get back to the next entry in the stack $ popd ~/.config/broot $ pwd /home/titus/.config/broot $ dirs ~/.config/broot
Another handy trick here is to
alias dirs="dirs -v"
so you get a nicer directory stack printout:$ dirs -v 0 ~/vd_agent 1 ~/entr 2 ~/.config/broot
-
Navigator keys
There's a fair few so I will list the best in no particular order
Ctrl-a
: go to the start of a lineCtrl-e
: go to the end of a lineAlt-f
: go forward one wordAlt-b
: go back one wordCtrl-w
: delete one wordCtrl-k
: delete everything after the cursorCtrl-u
: delete everything before the cursor
The bonus one is
Ctrl-xx
that jumps back to the start but remembers where you were. So jump to the start, change a flag, thenCtrl-xx
again to jump back to where you were.Alt-t
: swap the previous word with the current word.Ctrl-y
: can be used to paste things deleted withCtrl-[wku]
.
-
Ctrl-x Ctrl-e
This will pop your
$EDITOR
with the current command in the buffer so you can move around and edit in peace. Saving and quitting will then execute the command. -
Ctrl-z
,fg
,bg
, andjobs
I think most people know they can use
cmd &
to run a command in the background. Fewer people know you can usejobs
to list all of the commands you currently have running in the job stack (similar todirs
), and that you can then usefg
to bring the top command back into the foreground.Ctrl-z
is one of my personal favourites, for suspending the current command. Need to drop from a vim buffer to the terminal quickly?Ctrl-z
. Need to interrupt that long running process to just check the output file is in the right place?Ctrl-z
. After you have suspended a job you can send it to run in the background usingbg
, or bring it back into focus usingfg
.Both
fg
andbg
can take arguments to select which job you want to act on. You take the job number fromjobs
and then run e.g.fg %2
to bring job 2 to the foreground. -
watch
Keep an eye on some changing output. By default it will run a given command every two seconds and show the result, e.g.
watch ls -l
and see how new files come in and change.
Personal picks
These tools are things I am already using frequently on my personal machine. I added them to the alpine environment to demonstrate during the talk:
fzf
General purpose fuzzy finder. I've implemented clones of this program before for embedding in small applications, but fzf
continues to be a tool I used daily, particular for the Ctrl-R
override. Installation is normally via the package manager, and the shell completion and overrides can be grabbed from the repo and sourced to your .rc
file.
ugrep
For when you need to grep
you ugrep
. This tool saves me on so many occasions, there's really no reason not to use it. It has pretty much feature parity with grep
, but with incredibly good performance and very sensible additional options.
aria2
For when you really need to download some files off of the web and wget
just doesn't do it well enough. aria2
lets you download with multiple workers, which can massively speed up downloads, and will let you pick up interrupted downloads easily.
fd
fd
is a faster find
command. It also has colourful output, much like all of these modern tools, and the syntax is quite different but becomes natural with time. By default it will ignore hidden files, and can also be made to ignore files not checked into git.
fd thin # find paths containing the pattern `thin`
fd -g thin # --glob/-g find files named `thin`
fd
can also give you primitive parallelism in execution. For example, compressing many individual files, we can run xz
on each file in the current directory:
fd . -x xz
Alternatively we can execute something in batch, such as creating an archive of all files with a certain extension. For example, create a tarball of all *.fits
files in the data directory and call it obs.tar
fd . data -e fits -X tar cf obs.tar
The execution modes also support argument substitution for building more complex commands. Here we move all PDF files in the download directory to a collected directory, no matter how nested they are, keeping their names:
fd . -e pdf -x 'mv -nv {} ./collected/{/}'
We use -nv
for --no-clobber
to avoid overwritting and -v
for --verbose
to get a printout of each move.
The argument substitutions available are:
{}
: The path to the file (e.g./hello/world/thing.pdf
){.}
: The path without extension (e.g./hello/world/thing
){/}
: The basename (e.g.thing.pdf
){/.}
: The basename without extensions (e.g.thing
){//}
: The parent directory (e.g./hello/world
)
jq / yq / xq
Extremely useful for filtering json, yaml, and xml and making queries. These three are swissarmy knives for parsing output from various commands, logs, or endpoints. The flexibility that these tools give is incredible.
Setting up new tools
These were tools I'd heard about but never tried before.
z
z
is a shell script that tracks your most used and most recently visited directories, and adds a utility to quickly jump around and back to them based on a degree of pattern matching. This was the first tool I installed, and the installation is as simple as adding a line to your .bashrc
file. It then logs every time you cd
and updates its database.
I used it a little bit as I was playing around in the virtual machine. I must be honest I kept forgetting it was there, and would rely on muscle memory and fzf
to get around. When I did remember it was there, z
proved extremely reliable in getting me back to a place I had been doing some pluming in.
For how small the utility is, it's definitely worth having. It also stores all it's data in a plain text ~/.z
file, which means you can easily modify if it's learned something you don't want
eza
Like ls
but rainbow. Also has some genuinely extremely sensible filter and display options, such as "is this file included in Git?". Installation on Alpine (and many other places) is via the package manager, and the suggestion is to alias
alias ls="eza"
It doesn't do much more than that, except icons if your font set supports them. Just a solid alternative to ls
.
up - the ultimate plumber
Piping one command to another is useful, but sometimes you have a long running input into a malformed grep
and you need to rerun the whole thing. up
is supposed to make the piping interactive, and it does that really quite nicely; it lets you input a new command and waits for you to hit enter before running it and updating the output.
I found this particularly useful when piping selections, e.g. | sort | uniq | cut -d' ' -f 2
. Hitting Ctrl-X
saves the command to upN.sh
where N is an integer. This is fine, but it would be nice if it also printed the output buffer to stdout
so you have a record of what you just did, but it's a small scripting job I suppose to get that behaviour.
Installation is easy. Since it's written in go
you just get a single 4MB static executable.
broot
Broot is a directory navigator for the terminal. On first launch it will ask to install a script that lets you cd
from within broot
montblanc:~$ broot
Broot should be launched using a shell function.
This function most notably makes it possible to cd from inside broot
(see https://dystroy.org/broot/install-br/ for explanations).
Can I install it now? [Y/n]
y
Writing br shell function in /home/titus/.local/share/broot/launcher/bash/1.
Creating link from /home/titus/.config/broot/launcher/bash/br to /home/titus/.local/share/broot/launcher/bash/1.
/home/titus/.bashrc successfully patched, you can make the function immediately available with source /home/titus/.bashrc
The br function has been successfully installed.
You may have to restart your shell or source your shell init files.
Afterwards, you should start broot with br in order to use its full power.
The initial gripe with broot
is that it enables trees by default. You can toggle it with the :tree
command, and what I did was alias
alias broot="broot --no-tree"
alias br="broot"
I also modified ~/.config/broot/verbs.hjson
to add the Ctrl-[jkud]
vim keybindings.
Broot is packed with features. Some of the ones I think are quite interesting are
--whale-spotting
or-w
for quickly finding huge files or directories--dates
or-d
to sort by modified date-sdp
or--sizes
,--dates
, and--permissions
so that the output is familiar tols -l
There's a few quirks, which feels like broot can't decide if it's a navigator or filesystem explorer. For example, you need to remember to hit Ctrl-Enter
to cd
to the current directory. It feels like when you quit it should just take you to wherever you're in.
thefuck
The latest release isn't compatible with Python 3.12, so you have to install it from source which is nice and easy:
git clone https://github.com/nvbn/thefuck \
&& cd thefuck \
&& pip install .
Thefuck corrects commonly mistyped commands and suggests things that you probably wanted to type.
It works well from common commands but fails for less common commands:
$ fatsmod hello world
-bash: fatsmod: command not found
$ fuck
lsmod hello world [enter/↑/↓/ctrl+c]
It also can't work out that mr
might have been rm
, but it's occasionally useful. More of an amusing gimmick than something I would actually consider using.