7

Here is my use case: the command line utility melt can accept a file name, with the extension .melt on the command line, and open it; as an example, this is a proper test_p.melt file:

colour:blue
out=24

colour:red
out=48

... which opens and plays with melt test_p.melt.

Now, the thing is that .melt files do not support comments, which I wish they did (you'll get error messages for any line that contains an unparsable argument, including those with, say, a #). So here is a commented test_c.melt file:

# master comment here

colour:blue  # this is blue!
out=24

colour:red
out=48

Opening this in melt directly gives:

$ melt test_c.melt
Failed to load "# master comment here"
...

... and there is no blue screen shown.

So I thought - well, I can put in comments anyway, and then use Bash process substitution to filter the file with sed, and simply provide that to the melt application. First, tried a test with cat, which is successful:

$ cat <(sed 's/#.*$//' test_c.melt)



colour:blue  
out=24

colour:red
out=48

... looks good; but, if I try that with melt, it sees through my trickery:

$ melt <(sed 's/#.*$//' test_c.melt)
Failed to load "/dev/fd/62"
Failed to load "/dev/fd/62"

Basically, melt got the filename of the pipe Bash provided for the process substitution - but unfortunately, what melt does is that it processes argv[i] directly; and in case of a file, it needs to see a .melt extension in the filename; if it doesn't - the process fails.

So my question is: how could I use process substitution - so the filename of the pipe has a specific extension, in this case .melt? Basically, as a result of the substitution, I'd want a pipe filename of /dev/fd/62.melt, which I think will pass.

NB: of course, I can always do:

sed 's/#.*$//' test_c.melt > test_c_temp.melt
melt test_c_temp.melt

... but first, there are two commands here - and I'd want a one-liner pipeline; and for another, it opens up another problem of me thinking about removing temporary files afterwards, which I don't like.

Is this possible with Bash process substitution - or somehow with standard Linux tools?

  • 1
    Try melt < <(sed 's/#.*$//' test_c.melt) – Costas Jan 28 '15 at 21:55
  • Thanks @Costas - tried that, but apparently melt doesn't care about commands piped through stdin, since I get: Usage: melt [options] [producer [name=value]* ]+ and a dump of program options. Cheers! – sdaau Jan 28 '15 at 22:00
  • 1
    I have to read man melt on behalf of you. "If no files are specified, the compression is applied to the standard input". So sed 's/#.*$//' test_c.melt | melt > result.file can work. – Costas Jan 28 '15 at 22:07
  • Thanks again, @Costas - unfortunately, that doesn't work either, at least not on my version of melt (MLT melt 0.6.2; "standard input" is not mentioned in this version's man melt), since for that command, again I get Usage: melt [options] [producer [name=value]* ]+ and a dump. Cheers! – sdaau Jan 28 '15 at 22:10
  • I think you have to read man melt on your machine. If you cannot find answer post in on pastebin.com and post link here. – Costas Jan 28 '15 at 22:15
3

A possibility would be to point melt to a filesystem that shows modified copies of files. FUSE is a generic way to build filesystem driver implemented by an ordinary program and requiring no privileges. There are many FUSE filesystems around, and there's a good chance that one of them can help you. The idea is to provide a mount point where reading a .melt file reads the “real” file but with comments filtered out.

ScriptFS looks promising (but I've never used it). Something like this should work:

mkdir ~/uncommented-melt
scriptfs -p "$HOME/bin/uncomment-melt;&*.melt" ~/work ~/uncommented-melt

where ~/work is the root of the tree that contains your .melt files and ~/bin/uncomment-melt is

#!/bin/sh
sed 's/#.*$//' "$1"

Then if you have a file ~/work/test_c.melt with comments, you can run melt ~/uncommented-melt/test_c.melt.

Other potential helpful FUSE filesystems:

  • Execfuse — lets you build a simple FUSE driver with shell scripts
  • AVFS or other FUSE filesystems that transparently uncompress files: define the stripping of comments as an uncompression rule.
  • Many thanks for that @Gilles - I accepted this answer, because it answer the question generically. Unfortunately, I couldn't quite get scriptfs to work - here is a command log I saved at the time (AFAICR, stackexchange crashed that day :)) gist.github.com/anonymous/e9ee5e8c53d1b4c3c736 - so I ended up using a bash script approach which I posted below. Cheers! – sdaau Feb 23 '15 at 16:35
1

Ok, it turns out that in my specific case, these kind of melt scripts are to be interpreted strictly as commands line arguments; so just the one sed in the OP doesn't quite cut it (plus, there are other things like profiles that can be set). So here is what I ended up doing - can probably serve as inspiration in other cases which the title of the OP would cover.

I eventually settled on using this: make a test.shmelt file, which is actually a bash script that contains my commented melt script code; make this file executable chmod +x test.shmelt; then after editing the script, run it as ./test.shmelt.

Upon running, it will create a "cleaned up" test.melt file in /tmp, and call melt on this file instead. Since melt usually keeps running in terminal after the end of its programme, with a trap on SIGINT this temporary file can be cleaned up when Ctrl-C is pressed (but doesn't have to).

In that way, I still have comments; can edit quickly in the source file and run melt and see results; and have a file "cleaned up" of comments too, that I can use later.

Here is the code of test.shmelt:

#!/bin/bash

# call with: ./test.shmelt


#TMLTFILE="test.melt" # for final export
TMLTFILE="${0%%.shmelt}.melt" # for final export

function finished() { echo ; } ; # use this when testing or montaging to keep exported (tmp) .melt file; uncomment the below to remove the tmp file
#~ function finished() { rm /tmp/"$TMLTFILE"; echo "fin1 $0" ; } ; trap finished SIGINT ;

echo 'Remember `pulseaudio --start` to hear audio with `melt`!'
pulseaudio --check
if [ $? -ne 0 ]; then
    pulseaudio --start
fi

DIRNAME=$(readlink -f $(dirname $0))
PROFILE="square_ntsc"


echo "
# the profile doesn't work from here;
# still has to be specified on command line
# but including it here so the path is saved for testing
#~ -profile
#~ ${PROFILE}

-video-track

  # can avoid pixmap: here, but that s the producer;
  # qimage: also works
  # NB it is NOT '-out 1645'; but 'out=1645'!!
  /media/myimg/%05d.bmp
  in=0
  out=1645
  #~ length=1645
  #~ loop=0
  #~ eof=stop

-audio-track

  /media/mysnd/snd.wav
  in=0
  out=1645
  #~ loop=0
  #~ eof=stop

#-consumer xml # doesn't work here

" | sed -e 's/#.*$//' -e 's/^[[:space:]]*//' -e '/^[[:space:]]*$/d' -e 's/[[:blank:]]*$//' > ${TMLTFILE}

# the sed: remove comments; remove indents; remove empty lines; remove spaces at end of line


# eof: one of: stop, loop, continue or pause (eof=stop)
# to loop, use `melt eof=loop ...` (but make sure first,
#  that the edit as a whole stops - use xml to check!)
# due to caching issues, preview pieces (up to ~300 frames) like this:
melt eof=loop -profile ${PROFILE} ${TMLTFILE} in=200 out=400

# must have profile here, for checking w/ -consumer xml (else segfault)
# this command may add additional producers to the xml!:
## melt -profile ${PROFILE} ${TMLTFILE} -consumer xml

# use this to generate xml if needed:
#melt -profile ${PROFILE} $(cat ${TMLTFILE} | tr '\n' ' ') -consumer xml

# if exited normally, cleanup:
finished
1

It's been a while, but I'll try to answer the title question in a way that would've been useful to me.

This is related: Why does BASH process substitution not work with some commands?

So the specific problem I was trying to solve was this:

  1. I have an image converter;
  2. it converts from [insert random format] to simple formats like BMP/PNM/TGA;
  3. I want to convert to JPEG, so need to chain it with ImageMagick convert;
  4. I don't want to use temporary files.

So, unfortunately, the tool doesn't like to output to stdout/fds/pipes because it derives the output format from the output filename's extension. So how to trick it?

Create a soft link with the appropriate extension to /dev/fd/N and use that as the "temporary file".

Example:

ln -s /dev/fd/4 temp.pnm
[TOOL] -i [INPUTFILE] -o temp.pnm 4>&1 | convert - [OUTPUT.JPG]
1

With the zsh shell, you can use the =(...) form of process substitution (which uses temporary files instead of /dev/fd/x files and pipes) for which you can specify a suffix:

(TMPSUFFIX=.melt; melt =(sed 's/#.*$//' test_c.melt))

See info zsh TMPSUFFIX (assuming the info pages are installed, you may need to install a zsh-doc package) for details.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.