Command line tricks

I'm doing a little talk at my department's developer group on modern command line tricks, so though I would mirror that talk with a blog post about new terminal tools and speedy tips.

To avoid tools that pollute my environment too much I'm running all of these in a fresh Alpine Linux virtual machine.

montblanc:~/vd_agent$ uname -a
Linux montblanc 6.6.32-0-lts #1-Alpine SMP PREEMPT_DYNAMIC Fri, 24 May 2024 10:11:26 +0000 x86_64 Linux

I've been researching new tools and had a list I wanted to try out. So let's start at the top.

Shell zero overheads

Some things that you might not know that are very likely already on your machine and in your shell.

  • Ctrl-r

    Backwards search through your history.

  • cd -, pushd, popd and dirs

    Used to create a stack of directories. Useful for saving where you are and quickly coming back to it. For example, say I was in a certain directory and needed to get back to it after an accidental cd, then cd - will take me back to the last directory I was in:

    $ pwd
    /home/titus/.config/broot
    
    $ cd && pwd
    /home/titus
    
    $ cd -
    /home/titus/.config/broot

    The commands pushd and popd generalize this to a directory stack. You can view the stack at any time with dirs. Let's see how that works:

    $ pwd
    /home/titus/.config/broot
    
    # use pushd to add a specific else current directory to the stack
    $ pushd ~/entr/
    ~/entr ~/.config/broot
    
    # cd around a bit
    $ cd
    
    # check the directory stack
    $ dirs
    ~ ~/.config/broot
    
    # pop to get back to the next entry in the stack
    $ popd
    ~/.config/broot
    
    $ pwd
    /home/titus/.config/broot
    
    $ dirs
    ~/.config/broot
    Example push and pop directory

    Another handy trick here is to alias dirs="dirs -v" so you get a nicer directory stack printout:

    $ dirs -v
     0  ~/vd_agent
     1  ~/entr
     2  ~/.config/broot
  • Navigator keys

    There's a fair few so I will list the best in no particular order

    • Ctrl-a: go to the start of a line
    • Ctrl-e: go to the end of a line
    • Alt-f: go forward one word
    • Alt-b: go back one word
    • Ctrl-w: delete one word
    • Ctrl-k: delete everything after the cursor
    • Ctrl-u: delete everything before the cursor

    The bonus one is Ctrl-xx that jumps back to the start but remembers where you were. So jump to the start, change a flag, then Ctrl-xx again to jump back to where you were.

    • Alt-t: swap the previous word with the current word.
    • Ctrl-y: can be used to paste things deleted with Ctrl-[wku].
  • Ctrl-x Ctrl-e

    This will pop your $EDITOR with the current command in the buffer so you can move around and edit in peace. Saving and quitting will then execute the command.

  • Ctrl-z, fg, bg, and jobs

    I think most people know they can use cmd & to run a command in the background. Fewer people know you can use jobs to list all of the commands you currently have running in the job stack (similar to dirs), and that you can then use fg to bring the top command back into the foreground.

    An arbitrary job example

    Ctrl-z is one of my personal favourites, for suspending the current command. Need to drop from a vim buffer to the terminal quickly? Ctrl-z. Need to interrupt that long running process to just check the output file is in the right place? Ctrl-z. After you have suspended a job you can send it to run in the background using bg, or bring it back into focus using fg.

    Both fg and bg can take arguments to select which job you want to act on. You take the job number from jobs and then run e.g. fg %2 to bring job 2 to the foreground.

  • watch

    Keep an eye on some changing output. By default it will run a given command every two seconds and show the result, e.g. watch ls -l and see how new files come in and change.

Personal picks

These tools are things I am already using frequently on my personal machine. I added them to the alpine environment to demonstrate during the talk:

fzf

General purpose fuzzy finder. I've implemented clones of this program before for embedding in small applications, but fzf continues to be a tool I used daily, particular for the Ctrl-R override. Installation is normally via the package manager, and the shell completion and overrides can be grabbed from the repo and sourced to your .rc file.

An fzf example

ugrep

For when you need to grep you ugrep. This tool saves me on so many occasions, there's really no reason not to use it. It has pretty much feature parity with grep, but with incredibly good performance and very sensible additional options.

grep vs ugrep

aria2

For when you really need to download some files off of the web and wget just doesn't do it well enough. aria2 lets you download with multiple workers, which can massively speed up downloads, and will let you pick up interrupted downloads easily.

A small example downloading some XMM data

fd

fd is a faster find command. It also has colourful output, much like all of these modern tools, and the syntax is quite different but becomes natural with time. By default it will ignore hidden files, and can also be made to ignore files not checked into git.

fd thin              # find paths containing the pattern `thin`
fd -g thin           # --glob/-g find files named `thin`

fd can also give you primitive parallelism in execution. For example, compressing many individual files, we can run xz on each file in the current directory:

fd . -x xz

Alternatively we can execute something in batch, such as creating an archive of all files with a certain extension. For example, create a tarball of all *.fits files in the data directory and call it obs.tar

fd . data -e fits -X tar cf obs.tar
An fd example

The execution modes also support argument substitution for building more complex commands. Here we move all PDF files in the download directory to a collected directory, no matter how nested they are, keeping their names:

fd . -e pdf -x 'mv -nv {} ./collected/{/}'

We use -nv for --no-clobber to avoid overwritting and -v for --verbose to get a printout of each move.

The argument substitutions available are:

  • {}: The path to the file (e.g. /hello/world/thing.pdf)
  • {.}: The path without extension (e.g. /hello/world/thing)
  • {/}: The basename (e.g. thing.pdf)
  • {/.}: The basename without extensions (e.g. thing)
  • {//}: The parent directory (e.g. /hello/world)

jq / yq / xq

Extremely useful for filtering json, yaml, and xml and making queries. These three are swissarmy knives for parsing output from various commands, logs, or endpoints. The flexibility that these tools give is incredible.

Getting the current weather

Setting up new tools

These were tools I'd heard about but never tried before.

z

z is a shell script that tracks your most used and most recently visited directories, and adds a utility to quickly jump around and back to them based on a degree of pattern matching. This was the first tool I installed, and the installation is as simple as adding a line to your .bashrc file. It then logs every time you cd and updates its database.

I used it a little bit as I was playing around in the virtual machine. I must be honest I kept forgetting it was there, and would rely on muscle memory and fzf to get around. When I did remember it was there, z proved extremely reliable in getting me back to a place I had been doing some pluming in.

Navigating around with z

For how small the utility is, it's definitely worth having. It also stores all it's data in a plain text ~/.z file, which means you can easily modify if it's learned something you don't want

eza

Like ls but rainbow. Also has some genuinely extremely sensible filter and display options, such as "is this file included in Git?". Installation on Alpine (and many other places) is via the package manager, and the suggestion is to alias

alias ls="eza"
Eza and ls

It doesn't do much more than that, except icons if your font set supports them. Just a solid alternative to ls.

up - the ultimate plumber

Piping one command to another is useful, but sometimes you have a long running input into a malformed grep and you need to rerun the whole thing. up is supposed to make the piping interactive, and it does that really quite nicely; it lets you input a new command and waits for you to hit enter before running it and updating the output.

Using up to filter size and owner

I found this particularly useful when piping selections, e.g. | sort | uniq | cut -d' ' -f 2. Hitting Ctrl-X saves the command to upN.sh where N is an integer. This is fine, but it would be nice if it also printed the output buffer to stdout so you have a record of what you just did, but it's a small scripting job I suppose to get that behaviour.

Installation is easy. Since it's written in go you just get a single 4MB static executable.

broot

Broot is a directory navigator for the terminal. On first launch it will ask to install a script that lets you cd from within broot

montblanc:~$ broot

Broot should be launched using a shell function.
This function most notably makes it possible to cd from inside broot
(see https://dystroy.org/broot/install-br/ for explanations).

Can I install it now? [Y/n]
y
Writing br shell function in /home/titus/.local/share/broot/launcher/bash/1.
Creating link from /home/titus/.config/broot/launcher/bash/br to /home/titus/.local/share/broot/launcher/bash/1.
/home/titus/.bashrc successfully patched, you can make the function immediately available with source /home/titus/.bashrc

The br function has been successfully installed.
You may have to restart your shell or source your shell init files.
Afterwards, you should start broot with br in order to use its full power.

The initial gripe with broot is that it enables trees by default. You can toggle it with the :tree command, and what I did was alias

alias broot="broot --no-tree"
alias br="broot"

I also modified ~/.config/broot/verbs.hjson to add the Ctrl-[jkud] vim keybindings.

Broot is packed with features. Some of the ones I think are quite interesting are

  • --whale-spotting or -w for quickly finding huge files or directories
  • --dates or -d to sort by modified date
  • -sdp or --sizes, --dates, and --permissions so that the output is familiar to ls -l

There's a few quirks, which feels like broot can't decide if it's a navigator or filesystem explorer. For example, you need to remember to hit Ctrl-Enter to cd to the current directory. It feels like when you quit it should just take you to wherever you're in.

thefuck

The latest release isn't compatible with Python 3.12, so you have to install it from source which is nice and easy:

git clone https://github.com/nvbn/thefuck \
    && cd thefuck \
    && pip install .

Thefuck corrects commonly mistyped commands and suggests things that you probably wanted to type.

It works well from common commands but fails for less common commands:

$ fatsmod hello world
-bash: fatsmod: command not found
$ fuck
lsmod hello world [enter/↑/↓/ctrl+c]

It also can't work out that mr might have been rm, but it's occasionally useful. More of an amusing gimmick than something I would actually consider using.