Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I want to concatenate two files, i.e. staticEntries.dic and dynamicEntries.dic. As the names might show, the content of staticEntries.dic stays the same over time (contains common words), but to the contrary, dynamicEntries.dic might change over time (contains abbreviations that are extracted from DB). In my editor, I want to provide a mydict.dic, which contains the content of both files. I could do that via Shell, of course, but then I would have to replace the dynamicEntries.dic stuff from the mydict.dic file before concatenating a new version of dynamicEntries.dic to my resulting mydict.dic. -> I would like to create a mydict.dic file that contains a command to concatenate the two other files, but I don't know how to do that. Since it is not executed or called by my editor (I guess), I can't use bash commands.

Subsequently a simple example:

Content of staticEntries.dic

house
dog
horse

Content of dynamicEntries.dic. This content is generated based upon a database, so it changes over time.

EGB38
PD
UH7ZT

Pursued content of mydict.dic

house
dog
horse
EGB38
PD
UH7ZT

If I do that combination via shell, I have a problem if a new version of the file dynamicEntries.dic is generated: how to avoid duplicate entries in the mydict.dic file? So I would like to put the concatenation command in a file and reference it as a normal file I can provide as dictionary file to my editor. I don't know whether this is possible or if I have to apply another approach.

share|improve this question
    
What language are you using? Doesn't it have an include function? –  Bernhard Sep 15 '13 at 11:36
    
At the moment no language?! A dic file is a static, non interpreted file so how can I use a language? Did I overlook something? –  strauberry Sep 15 '13 at 11:37
1  
Almost any editor worth using that has dictionary support should allow you to specify multiple dictionaries. What editor are you using and have you checked its documentation? –  msw Sep 15 '13 at 12:05
    
What are you trying to accomplish? How are you going to use mydict.dic? What tools will you use? It's hard to solve your problem without that knowledge. –  Bananguin Sep 16 '13 at 7:54
add comment

5 Answers

up vote 7 down vote accepted

You could possibly write a script that sits behind a named pipe and dumps the contents of both staticEntries.dic and dynamicEntries.dic whenever it's opened and read from. Take note of the pipe being closed and terminate output until it is opened again.

But you'd have to leave that script running in the background, and remember to start it up again after logout/login or reboot.

More importantly, it is not a novice shell programming task.

Sometimes (usually), the simplest solution is best.

It is far simpler to just create a Makefile that defines mydict.dic as being dependant on the other two files and remembering to run make to update it when you need it. or just a shell script - the advantage of a Makefile is that you could also run it from cron and it would only update the target file (mydict.dic) if either of the source files had changed.

for example:

#!/usr/bin/make -f

all: mydict.dic

mydict.dic: staticEntries.dic dynamicEntries.dic
        cat staticEntries.dic dynamicEntries.dic > mydict.dic.tmp
        mv mydict.dic.tmp mydict.dic

the lines with cat and mv start with a tab, not spaces.

The concatenated file is created as a tempfile first and then moved into place, so the replacement of the old with the new is an atomic operation. this is done so that whenever you use the file, you have either the complete old version or the complete new version, but never a partial version of the new.

if either of the source .dic files are in a different directory, you'll need to specify the full pathnames to the files.

share|improve this answer
    
BTW, you can insert a pipe to sort -u in the cat line, immediately before the > redirection to create a sorted, unique list. or just use sort -u instead of cat. –  cas Sep 15 '13 at 12:01
    
+1 for the pipe idea, should usually work if the reader just loads the whole thing in one go, rather than trying to seek around in it... –  frostschutz Sep 15 '13 at 12:19
    
Where one should actually write cat $^ >$@.tmp and mv $@.tmp $@. On second thought the mv only has a minor advantage over a simple cat $^ >$@. Because replacement is also bad if the file is currently read, so you would estimate quite a long execution time of cat before the mv solution really gets to show its advantages. –  Bananguin Sep 16 '13 at 8:02
    
I deliberately wrote the Makefile to be as simple and explicit as possible, i wanted it to be easy to understand even for someone who'd never written or seen a Makefile before. Also, the atomic replacement with mv is still important - if the old version of file is currently being read, i.e. if it is open, then the process that has it open will continue to read from the old version until the file handle is closed (at which point, the old version will finally be completely deleted) –  cas Sep 17 '13 at 3:38
add comment

I would just run a little cronjob that re-concatenates the file if the dynamic dictionary has been modified. First, write a little script that watches the file for changes:

#!/usr/bin/env bash
while true; do
  inotifywait -e modify path/to/dynamicEntries.dic
  sort path/to/dynamicEntries.dic /path/to/staticEntries.dic | 
    uniq > mydict.dic.tmp
  sleep 1
done

Now, save that script as dict.sh or something, make it executable (chmod a+x dict.sh) and make a crontab that launches it script on reboot. Run crontab -e and paste in this line:

@reboot /path/to/dict.sh

Now, the dictionary should be automatically updated every time that the dynamic one changes and you don't need to run anything manually.

share|improve this answer
    
What happens if it's modified again during the sort/sleep? Isn't it more reliable to run it in --monitor mode and read from it? –  frostschutz Sep 15 '13 at 15:53
    
@frostschutz I am assuming it does not get modified all that often since the sort and sleep should only take seconds. Not an expert on inotify though so you may well be right. –  terdon Sep 15 '13 at 15:57
add comment

Concatenating files in the shell is easy:

cat staticEntries.dic dynamicEntries.dic

This will print the contents of those two files on stdout. If you want to redirect the result into a new file, do:

cat staticEntries.dic dynamicEntries.dic > mydict.dic

But this is probably not what you're looking for...? I have to say, I don't really understand the problem that you're trying to solve.

share|improve this answer
    
*The cat program is a standard Unix utility that concatenates and lists files. The name is an abbreviation of catenate, a synonym of concatenate :) [en.wikipedia.org/wiki/Cat_(Unix)] –  Chirp. Not Luke. Sep 15 '13 at 11:57
add comment

If you want a file to be dynamically concatenated every time you access it, you'd probably have to resort to writing a FUSE filesystem or similar mechanism for it.

Alternatively you could map them using loop devices / device mapper but I doubt this would work out properly. In particular the mapping would have to be udpated every time the files change.

If whatever program is reading this file is open source, it might be easier to patch it to read both files in the first place. Or wrap it in a shell script so the dic file is concatenated every time you start your program.

share|improve this answer
    
FUSE? I wish I could give this answer a "swatting flies with dynamite" badge :) –  msw Sep 15 '13 at 12:09
3  
Maxim 37: There's no such thing as overkill, there's only shoot and reload. ;) It happens when the question presents a questionable solution in the first place. –  frostschutz Sep 15 '13 at 12:21
    
Upvoting based on amusement value is probably contrary to SE intentions, but I do intend to cite your Maxim 37 in the future. –  msw Sep 15 '13 at 12:27
    
+1 for Maxim 37 –  Bananguin Sep 16 '13 at 8:11
add comment

You can change the way you call your editor. Assume you are using vi , you can create a shell script editmydictionary.sh:

cat staticEntries.dic dynamicEntries.dic > mydict.dic.tmp
vi mydict.dic.tmp

so whenever you execute editmydictionary.sh, you are editing the latest version of the 2 files.

share|improve this answer
    
+1 Interesting idea. I'm not experienced enough with vi scripting, but do you think this can be implemented from within .virc so that it applies to any file you open with vi? –  Joseph R. Sep 15 '13 at 19:59
    
Or vi <(cat staticEntries.dic dynamicEntries.dic) if using bash shell. –  Jakub Narębski Sep 15 '13 at 20:21
1  
@JosephR.: I don't know about vi, but vim sure can. You need autocommand and trigger a function on newbuf. –  Bananguin Sep 16 '13 at 8:09
    
@Bananguin Thanks. I'll look into it. –  Joseph R. Sep 16 '13 at 8:47
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.