1

It seems that each core Linux command has a different stdout format. Also, these formats are not always easily/safely/consistently parsed by other scripts/apps.

Is there a wrapper or specification or library that provides a unified/consistent output for easy parsing (like JSON or UniqueName:Value encoding)?

Example 1: ps -A

    PID TTY          TIME CMD
    558 tty1     00:00:00 startx
    576 tty1     00:00:00 xinit
    577 tty1     00:00:37 Xorg
    590 tty1     00:00:01 awesome
   8281 pts/0    00:00:00 ps

Example 2: lshw / lscpu

While lshw is key:value output, but very hard to parse into unique key:value pairs.

cedar
    description: Computer
    width: 64 bits
    capabilities: smp vsyscall32
  *-core
       description: Motherboard
       physical id: 0
     *-memory
          description: System memory
          physical id: 0
          size: 7936MiB
     *-cpu
          product: Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz
          vendor: Intel Corp.
          physical id: 1
          bus info: cpu@0
          size: 2611MHz
          capacity: 3GHz
<SNIP>

Example 3: ls -l

total 36
drwxr-xr-x  2 guy guy 4096 Nov 19 08:41 Desktop
drwxr-xr-x  2 guy guy 4096 Dec 26 13:37 Downloads
drwxr-xr-x  3 guy guy 4096 Nov  7 19:39 go
drwxr-xr-x  6 guy guy 4096 Jan 15 12:42 play
drwxr-xr-x 12 guy guy 4096 Jan 16 19:27 repo
drwxr-xr-x  3 guy guy 4096 Oct 15 18:39 RiderProjects
drwxr-xr-x  8 guy guy 4096 Jan 13 17:38 scripts
drwxr-xr-x 12 guy guy 4096 Jan 10 16:48 source
drwxr-xr-x  6 guy guy 4096 Jan  4 14:31 temp

Example 4. file /bin/bash

/bin/bash: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=6c75f9f0f273cf6549f078b042c0a3f5a04f0357, for GNU/Linux 4.4.0, stripped

Alternative: Common shell Object-Model

Some shells like PowerShell Core have a common internal object model for consistent data paring between functions. Is there something similar of linux core utils?

Related Questions

Useful comments

7
  • 2
    There is no such thing as far as I am aware of. Many commands have parameters you can pass to format the output. You can then use tools like grep,cut, tr, awk, sed, etc to further format the output the way you want.
    – GMaster
    Jan 17, 2022 at 10:46
  • 1
    Note that none of lshw, lscpu, ps are part of GNU coreutils and GNU's Not Unix, but neither it is Linux. On most Linux-based systems, core utilities are from toybox (Android) or busybox (most others). Only desktop/server Linux-based OSes tend to use GNU utilities, but so does Cygwin, and GNU utilities can be built and used on most other systems. Jan 17, 2022 at 10:46
  • 2
    Does this answer your question? How to encode different data types for STDOUT so that STDIN can identify which is what?
    – Kusalananda
    Jan 17, 2022 at 11:21
  • @they, json can only hold Unicode text data, so you can't even have file paths in there (which are sequences of non-NUL bytes which don't have to form text in the current locale character encoding or UTF-8) except by using some encoding or some extensions over the JSON standard, so JSON may not be the best choice. There is a growing list of utilities that have started going that way though. FreeBSD as well. Jan 17, 2022 at 12:15

3 Answers 3

2

As pointed out by Taqras, the simple answer here is "no". Historically "the" API for *nix operating systems has be the C API (as documented in man 2 instead of man 1). Just about the only place a common core shell interface would be really useful would be in languages which cannot access the C API... and by that I mean in shell scripts.

The primary function of the shell is to be a user interface.

Inevitably the shell has become much more than a simple UI. It's very useful to be able to glue commands together, fetching arguments from one to use in another... But by the time that you are looking for robust parsing and common interfaces, it's time to move on to full fledged programming language that's been designed as a programming language from the start.

In short there's just not the market pressure to create a common shell interface.

Those who want quick and simple scripts to get something done with a tiny amount of code will use the shell (Bash etc.). Where as those who want to write robust programs with good API tend to steer towards languages like Python or compiled languages like Go or C.


Incidentally parsing the output of ls is generally discouraged. It's better to use globbing within the shell itself.

1

The "common internal object model"-thing isn't really a shell property; it's a property of the underlaying operating system.

In Windows you have stuff like CIM or WMI.

In Linux commands tends to output plain text, more or less orderly formatted - and, as @GMaster commented, there are tools available to process that outputted text. This means you'll typically pipe commands together, and alias them to something short, if it's something you do often.

0

You're assuming that all these utilities were written approximately at the same time, or by the same people, or with the intent of being easily parsable. That's not the case. These utilities produce vastly different types of information, so no one ever thought of unifying their output. Lastly, XML/JSON and other formats are relatively new while the original Unix utilities were written over 30 years ago at the time when the processing power was low and storage cost a fortune. XML and JSON are both ~21 years old.

Is there a wrapper or specification or library that provides a unified/consistent output for easy parsing (like JSON or UniqueName:Value encoding)?

No one has done that to my best knowledge. Patches are welcome.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .