25

I've noticed a lot of questions and answers and comments expressing disdain for (and sometimes even fear of) writing scripts instead of one-liners. So, I'd like to know:

  • When and why should I write a stand-alone script rather than a "one-liner"? Or vice-versa?

  • What are the use-cases and pros & cons of both?

  • Are some languages (e.g. awk or perl) better suited to one-liners than others (e.g. python)? If so, why?

  • Is it just a matter of personal preference or are there good (i.e. objective) reasons to write one or the other in particular circumstances? What are those reasons?

Definitions

  • one-liner: any sequence of commands typed or pasted directly into a shell command-line. Often involving pipelines and/or use of languages such as sed, awk, perl, and/or tools like grep or cut or sort.

    It is the direct execution on the command-line that is the defining characteristic - the length and formatting is irrelevant. A "one-liner" may be all on one line, or it may have multiple lines (e.g. sh for loop, or embedded awk or sed code, with line-feeds and indentation to improve readability).

  • script: any sequence of commands in any interpreted language(s) which are saved into a file, and then executed. A script may be written entirely in one language, or it may be a shell-script wrapper around multiple "one-liners" using other languages.


I have my own answer (which I'll post later), but I want this to become a canonical Q&A on the subject, not just my personal opinion.

  • 10
  • 4
    Regarding Python, it's generally bad for one-liners because whitespace is part of the syntax, including indentation and newlines. As well, it's more verbose and explicit than most other common languages. – wjandrea Sep 30 '19 at 23:07
  • 2
    Anything I've had to google (some regex, typically) and may reuse in some shape or variant, I'll save in a script file in cloud (dropbox, google cloud, whatever). The commentary contains the keywords I know I'm going to use when I'll need it again. It saves 5+minutes re-weighing my choices between various similar answers, and avoiding pitfalls, or constructing the variant needed as I didn't find the exact well-written variant I needed. – user3445853 Oct 1 '19 at 11:20
  • 3
    @wjandrea To add: regexes. Perl has a specific syntax for them that includes the operation to perform etc. In python you need an import and write function calls with string literals as arguments which takes a lot more characters. Most one-liners use plenty of regexes to manipulate text so this is a "big issue". – Giacomo Alzetta Oct 1 '19 at 14:34
  • 1
    @wjandrea Python isn't bad for one-liners, not in the sense of not-being-able to write one. On average I've been able to transform 4-5 line scripts into one-liners. But as you mentioned, it's more explicit than other languages (which IMHO it's one of the advantages of Python). So they can get into double or triple character count than say Perl or awk (which also don't need import some libraries unlike Python), and that's what most people complain about - the length of those one-liners. But then again, IMHO it's petty to argue about character count; if something works - it works – Sergiy Kolodyazhnyy Oct 2 '19 at 4:30

10 Answers 10

33

Another response based on practical experience.

I would use a one-liner if it was "throw away" code that I could write straight at the prompt. For example, I might use this:

for h in host1 host2 host3; do printf "%s\t%s\n" "$h" "$(ssh "$h" uptime)"; done

I would use a script if I decided that the code was worth saving. At this point I would add a description at the top of the file, probably add some error checking, and maybe even check it into a code repository for versioning. For example, if I decided that checking the uptime of a set of servers was a useful function that I would use again and again, the one-liner above might be expanded to this:

#!/bin/bash
# Check the uptime for each of the known set of hosts
########################################################################
#
hosts=(host1 host2 host3)

for h in "${hosts[@]}"
do
    printf "%s\t" "$h"
    uptime=$(ssh -o ConnectTimeout=5 -n "$h" uptime 2>/dev/null)
    printf "%s\n" "${uptime:-(unreachable)}"
done

Generalising, one could say

  • One-liner

    • Simple code (i.e. just "a few" statements), written for a specific one-off purpose
    • Code that can be written quickly and easily whenever it is needed
    • Disposable code
  • Script

    • Code that will (probably) be used more than once or twice
    • Complex code requiring more than "a few" statements
    • Code that will need to be maintained by others
    • Code to be understood by others
    • Code to be run unattended (for example, from cron)

I see a fair number of the questions here on unix.SE ask for a one-liner to perform a particular task. Using my examples above, I think that the second is far more understandable than the first, and therefore readers can learn more from it. One solution can be easily derived from the other so in the interests of readability (for future readers) we should probably avoid providing code squeezed into in one line for anything other than the most trivial of solutions.

  • That's a nice practical example of how a quick-and-dirty one-liner can evolve into a more robust script. – Anthony Geoghegan Sep 30 '19 at 11:17
  • This is a good start, but there's one other possibility. I have some things that are handy to use more than once or twice, but probably not on the same machine. I save them in a text file I keep open at all times on my work computer, so that I can quickly copy them and paste into my SSH client (or a command prompt on a Windows server, for that matter). These are not necessarily "one liners" nor do I go through the effort of saving them in files as "scripts". I call them "recipes". – Monty Harder Sep 30 '19 at 19:55
  • @MontyHarder For such cases I tend to save them as scripts then store them in a git repo. I can then simply install them on any machine I need via a simple git clone. That is for work. For personal use I never write code in a "recipe" file (I don't use github snippets for example). I save all my scripts in a $HOME/bin directory that I've kept since my college days in 1997 (on the universitys SunOS). I got the idea from the novel Neuromancer where both the protagonist and antagonist kept personal scripts/programs to use as weapons – slebetman Sep 30 '19 at 21:22
  • 1
    Though it’s certainly tangential to the actual discussion, in your example script you could use set -- host1 host2 host3 then for h in "$@" (or just for h), which would make the script POSIX-portable. :-) – wchargin Oct 1 '19 at 6:49
  • 2
    I like the point about making code more readable to OP and viewers to learn. Best to maybe do both in an answer, but I find it easier to concatenate a script vs deconstruct a one liner personally. – FreeSoftwareServers Oct 3 '19 at 8:03
10

Write a script when:

  • more code is required
  • you value readability
  • it is necessary/useful to add comments, to show what the code does
  • you need to pass parameters
  • you want the script to run in its own environment (variables, etc.)
  • you are probably going to reuse/adapt the code for some more complex purpose

Write a one-liner when:

  • only a small amount of code is necessary
  • you want to access variables defined in the current shell
  • you need a quick & dirty solution
  • It's usually easy to modify what things a one-liner operates on. That "pass parameters" point for a script maybe should be "you want to package it up for simple use later". I've written "one-liners" that include defining a shell function and calling it. Although now that I think about it, that one has settled down after several iterations and I should put it in a script. – Peter Cordes Oct 3 '19 at 11:16
9

When a required series of commands is fairly short and/or produces a result that could be usable as a part of a larger pipeline or a command alias, it might be a good one-liner.

Basically, I think one-liners are usually things that a seasoned system administrator familiar with both the problem and the commands used might write on the spot, without too much thinking.

When an one-liner becomes excessively long or involves more than very simple conditionals or loops, it is usually better to write them as a multi-line script or shell function, for readability. Also, if it's something you write to be used again and again, or something that will be used (and possibly troubleshot) by other people, you should be willing to spend the time to write the solution into a clear(er), commented, script form.

In Python, indentation is part of the syntax, so you literally cannot use the features of the language to their full extent if you write a one-liner.

Perl and awk one-liners are often about using the raw power of regular expressions. But some call regular expressions a Write-Only Language, not entirely without reason. Writing a multi-line script allows you to write in comments to describe what a particular regular expression is supposed to be doing, which will be very helpful when you look at the script again, after doing other things for six months in between.

This is inherently a very opinion-based question, as it involves gauging what amount of complexity is acceptable and tradeoffs between readability and compactness; all of these are very much matters of personal judgement. Even the choice of a language can depend on personal matters: if a particular language would theoretically be optimal for a particular problem, but you're unfamiliar with it, and already know how to solve the problem with some other language, you may choose the familiar language in order to get the job done quickly and with minimum effort, although the solution might technically be somewhat inefficient.

The best is often the enemy of good enough, and this applies very much here.

And if you are familiar with Perl, you may have already heard of TMTOWTDI: There is More Than One Way To Do It.

  • plus one for mentioning "or shell function" – glenn jackman Oct 2 '19 at 17:28
7

Please note that this is a personal opinion; take it with a grain of salt.

  1. If the command fits within, say, 80 characters, use a one-liner. Long command lines are hard to work with. There's also the problem of recurrence, see #2

  2. If you need to run the command(s) more than once, use a script. Else, use a one-liner, provided condition #1 is met.

  3. This is very subjective, but yes, I see shell, Perl or awk as being my go-to tools for one-liners.
  4. See #1, #2, #3.
  • 3
    I'm liking the answers i've seen so far. I especially like your 1st point - editing is one of the things I was planning to mention in my answer - even with ^X^E, it's much easier and far more pleasant to edit a script in your favourite editor than edit on the command-line. – cas Sep 30 '19 at 10:29
  • 2
    Upvoted though point 4 seems kind of pointless. :) – Anthony Geoghegan Sep 30 '19 at 16:03
  • @cas: with control-arrow-key (or alt+f / alt+b) you can move around in a one-liner very fast. Especially if you have your key auto-repeat settings at like 50/sec with a short-ish delay like 220ms. You can also use control-s / control-r isearch within a one-liner, although that can take you back into other history. If you're good with control-w, alt+backspace, alt+d, and control-y, you can do a lot with just keyboard cursor movement. – Peter Cordes Oct 3 '19 at 10:35
  • Also, IIRC some shells (like ZSH I think) have a keybind to invoke an editor on the current command line, allowing you to compose a "one-liner" in your favourite editor and easily get that into command-history for further up-arrow and editing. (Being able to modify it for re-runs is what makes one-liners great, vs. handling command-line options in a script.) IIRC, bash has external edit, too, but not bound to anything by default. – Peter Cordes Oct 3 '19 at 10:37
  • @PeterCordes moving the cursor very fast doesn't even begin to compare with what you can do in a decent editor like vi/vim. btw, ^X^E invokes $EDITOR or $VISUAL in bash (and some other shells. zsh too, i think. configurable). it's a good way of turning the unreadable mess I've created into something comprehensible by inserting line-feeds and indentation - it's easy to get lost in nested braces or brackets without them. BTW my terminal windows are over 250 characters wide, so my one-liners can get very long even without wrapping. – cas Oct 3 '19 at 12:46
7

I view and use a one-liner as a temporary development tool to edit repeatedly until it does what I want, or I understand how some subtlety of a sub-command works.

It then either gets noted (and annotated) in a text file on the subject I was investigating, or gets cleaned up into a script that gets put in my personal bin PATH, possibly for further refinement, parameterisation and so on. Typically, the one line will be broken up to more readable, even if no other changes are made, or I never use the script again.

I set my shell history to be over 5000 lines, with a separate history per terminal, and per login on a remote and so on. I hate not being able to find in these histories a one-line command I developed a few weeks back and didn't think I would need again, hence my strategy.

Sometimes, a whole group of commands are need to do something, like configure some hardware; at the end I copy them all out of the history, clean them up minimally, and add them to a shell script RUNME just as notes on what I did, but also so that they are almost ready to be used again by someone else. (This is why I find tools that only provide a GUI to configure them such a pain.) I find this an incredible gain in efficiency, as so often something you expect to do only once, has to be then done another 5 times...

  • 1
    +1 one-liners are great for up-arrow and edit, vs. implementing command-line options for a script to control what it does separately from what it operates on. e.g. change ln vs. ln -s for hard vs. soft links in a one-liner that loops over some files. – Peter Cordes Oct 3 '19 at 11:09
5

I would want the people in my team to write scripts for whatever operations that may be required to be performed more than once (i.e. "workflows" or "pipelines"), and for whatever one-off task that is required to be documented (typically things like "modify certain things in some dataset to fix incorrectly supplied data" in our case). Apart from that, "one-liners" or scripts don't matter too much.

Failing to properly document workflows (as scripts or equivalent) makes it harder to introduce new staff on a project, and also makes it harder to transition people out of a project. It additionally makes any type of auditing difficult, and tracking down bugs may be impossible.

This is regardless of what language is used for programming.

Other workplaces may have similar/different customs or expectations, maybe even written down.

At home, you do whatever you feel like.

For example, I write scripts as soon as I do something non-trivial in the shell, like before submitting an answer to an U&L question, or whenever I need to test the individual components of a command or set of commands before running it "for real".

I also write scripts for repetitive tasks that I really want to do the same way every time, even if they are simple, like

  • updating my OpenBSD system and all installed packages (this comes down to three short commands, and a handful of others for housekeeping, collected in a short script), or
  • backing up my system to an off-site storage (essentially a single command, but again with quite a lot of housekeeping and the script also allows me to do diffs between snapshots etc., and it needs to work subtly different on different systems, and I run it from a cron job), or
  • fetching mail (again, essentially a single command, but I want it to behave differently when called as a cron job and when I use it from the command line, and it shouldn't try to fetch mail under some circumstances, and it writes a short log message to a file).

I use shell functions (very seldom aliases) for convenience in interactive shells, e.g. to add options to certain commands by default (-F for ls) or for creating specialised variations of some commands (pman here is a man command that only looks at POSIX manuals, for example).

5

Looks like I hold a minority view on this.

The term "script" is intentionally confusing, get rid of it. This is software you're writing there, even if you're using solely bash and awk.

Within a team, I prefer to write software with code-review process enforced by a tool (github/gitlab/gerrit). This gives version control as a bonus. If you have multiple systems, add continuous deployment tools. And some test suites, if the target systems are important. I am not religious about these, but weigh the benefits and costs.

If you have a team of admins, the simplest vim /root/bin/x.sh is mostly a disaster in terms of change-related communication, code readability, and cross-system distribution. Often one serious malfunction costs more time/money than the supposedly "heavy" process.

I prefer one-liners actually for anything that doesn't deserve that "heavy" process. They can be reused rapidly, with a few keystrokes, if you know how to use your shell history effectively; easily pasted between unrelated systems (e.g. different customers).

Crucial difference between (unreviewed!) one-liners and the reviewed software ("scripts"): the responsibility is fully on you when you execute one-liners. You can never say to yourself "I just found this one-liner somewhere, I ran it without understanding, what followed is not my fault" - you knew you are running unreviewed and untested bit-of-software, so it is your fault. Make sure it's small enough that you can foresee the results - be damned if you don't.

You need this simplistic approach sometimes, and setting the bar at about one line of text works well in practice (that is, the boundary between reviewed and unreviewed code). Some of my one-liners look terrifyingly long, but they do work effectively for me. As long as I grok them and I don't shove it onto more junior colleagues, it's all fine. The moment I need to share them - I go through standard software-development process.

  • 1
    good point: I like the term "program" over "script" myself. – glenn jackman Oct 2 '19 at 17:31
5

I must say I'm somewhat horrified by the implications of the first parts of the question. The Unix scripting languages are full-fledged programming languages, and programming languages are languages, meaning that they're just about as infinitely malleable and flexible as human languages. And one of the hallmarks of "infinite malleability and flexibility" is that there's almost never "one right way" of expressing something -- there are varieties of ways, and this is a good thing! So, yes, of course it's a matter of personal preference, and there's nothing wrong with that. Anyone who says there's only one way of doing something, anyone who expressed disdain about doing (or not doing) something one way or the other is probably being a pedant or a prescriptivist.

Once upon a time, my own ~/bin was full of "useful" little scripts I'd written. But I realized that most of them got used the day I wrote them, and never again. So the more time that goes by, the smaller my bin directory gets.

I don't think there's anything wrong with writing a script. If you imagine that you or someone else might use it again, by all means, write a script. If you want to "properly engineer" a supportable script for the sake of it, by all means, do so. (Even if nobody ever uses it, writing it is good practice.)

Reasons I don't write scripts as much any more (and, I guess, favor "one-liners"):

  • bin directory clutter does have a cost. I don't put things there any more unless they really belong there.
  • The scripting languages I use are languages that I use; I'm really facile with them by now. Retyping a one-liner whenever I need it doesn't feel like work; I don't feel a mandate to keep and save and use a script just to relieve the (re)typing burden. (Among other things, at some point the retyping burden becomes less than the memory burden of remembering what a script is.)
  • Another reason retyping things doesn't feel like a burden is: command history. There are plenty of one-liners that I haven't enshrined as scripts, even though I use them every day -- but I don't have to retype them; I just recall them from history. In that way they're sort of poor-man's scripts or aliases.
4

I believe that most of the time the request for a "one-liner" is in fact a request for a "sound bite", that is:

In the context of journalism, a sound bite is characterized by a short phrase or sentence that captures the essence of what the speaker was trying to say, and is used to summarize information ...

People want and request for the shortest code that demonstrates a solution to the question asked.

Sometimes, there is no way to reduce an speech to a "sound bite" as well as may be impossible to reduce an script to a short "one-liner". In such cases the only possible answer is an script.

And, in general, a "sound bite" is never a good substitute of the whole speech. So, a "one liner" is generally only good to convey an idea or to give a practical example of that idea. Then, after the idea is understood, it should be expanded (applied) in an script.

So, write a "one-liner" to convey an idea or concept.

Write your general code as full scripts not one-liners.

1

You ask about "fear and disdain of scripts". This is due to the Q+A situation, where often the answer is saying: It needs this step and this, but I can also put it in one line (so you can copy paste it and use it as a long simple command).

You can sometimes only guess if the Q is better served by a quick and dirty copy paste solution, or if a "nice" script is exactly what the Q needs to understand.

A lot of the pros and cons here are true, but still they are missing the point. I would even say the definitions in the Q are not helpful.

The shell history files are "one liners", but still they are saved. The bash function operate-and-get-next (C-o) even lets you repeat them semi-automatically.

I hardly see the word function mentioned. That is the way to store both "scripts" and "one liners". And with a few keystrokes you can switch between both formats. A compound command can often fit both.

history -r file is the way to read one or many command lines (and comments) from any file, not just from default history file. It just takes some minimal organizition. You shouldn't rely on a large history file size and history search to retrieve (some) version of a command line.

Functions should be defined in a similar way. For starters, put thisfunc() {...} in a file "functions" in your home dir, and source it. Yes this is some overhead, but it is the general solution.

If you store every command line and every script variation in a separate file, it is a (theoretical, but objective) waste of filesystem blocks.

A one-liner has nothing quick and dirty by itself. It's more the copy-paste part that is bit frowned upon, and how we just rely on command history to save it for us.


@sitaram: your one liner really has un-onelinerish features: shebang line, newline, four utility calls - two of them sed "programs". I think the original perl script looked nicer and ran faster and did not waste any bytes by spreading out on 60 lines. And perl also lets you squeeze quite a bit.

For me, that is exactly the kind of one liner to avoid. Would you want to have that (pasted) on your command line? No, and then, if you store it in a file anyway (no matter script or function), you might as will invest two or three newline-bytes to make it look --- nice.


10 min. later: I just wanted to check if I can say "two sed programs" or of it is "two sed substitutions/calls". But sitaram's answer is gone. My reply still makes sense I hope.

Not the answer you're looking for? Browse other questions tagged or ask your own question.