1

I have downloaded about 3200 websites to the depth 2. So now I have one master folder (abc) that contains many folders, containing files for each website. So my folder abc contains 3200 folders and each folder contains other folders that contains files with text from the websites. I have also a script that can edit text in each file. It is stored in file named lynx.sh:

#!/bin/bash
fileA=$1
while IFS= read -r lineA
do
LTRA=$(echo "${lineA:0:1}")
catA=$(lynx -dump -nonumbers -nomargins -nolist -noprint -width 1000 -assume_charset=utf-8 $2/*.* )
editA=$(echo "$catA" | sed -e 's/\[[^][]*\]//g')
editB=$(echo "$editA" | sed -e 's/\s\+/\n/g')
editC=$(echo "$editB" | sed '/^http/ d' )
editD=$(echo "$editC" | sed '/^IFRAME/ d' )
editE=$(echo "$editD" | sed 's/<[^>]*>//g' )
editF=$(echo "$editE" | sed -r 's/[^aáäbcčdďdzdžeéfghchiíjklĺľmnňoópqrŕsštťuúvwxyýzžAÁÄBCČDĎDZDŽEÉFGHCHIÍJKLĹĽMNŇOÓPQRŔSŠTŤUÚVWXYÝZŽ][^aáäbcčdďdzdžeéfghchiíjklĺľmnňoópqrŕsštťuúvwxyýzžAÁÄBCČDĎDZDŽEÉFGHCHIÍJKLĹĽMNŇOÓPQRŔSŠTŤUÚVWXYÝZŽ]+//g' )
editG=$(echo "$editF" | sed s'/[^aáäbcčdďdzdžeéfghchiíjklĺľmnňoópqrŕsštťuúvwxyýzžAÁÄBCČDĎDZDŽEÉFGHCHIÍJKLĹĽMNŇOÓPQRŔSŠTŤUÚVWXYÝZŽ]$//')
editH=$(echo "$editG" | sed s'/^[^aáäbcčdďdzdžeéfghchiíjklĺľmnňoópqrŕsštťuúvwxyýzžAÁÄBCČDĎDZDŽEÉFGHCHIÍJKLĹĽMNŇOÓPQRŔSŠTŤUÚVWXYÝZŽ]//')
editI=$(echo "$editH" | sed 's/ .*//')
editJ=$(echo "$editI" | sed '/^$/d' )
echo "$editJ" > $2/"blaaa"_lynx.txt
echo "$lineA"
done <"$fileA"

It makes text in each file to be edited - every word is on the new line. I have used this script many times before but I use it with file input.txt which contains names of all websites that i have.

Now I am trying to edit all folders in my abc folder at once. I have tried to use somtehing like this:

find /home/student/eny/abc -exec lynx.sh {} \;
find /home/student/eny/abc/* -iname -exec ./lynx.sh input.txt {} \;

and many others. I can not find a solution for this.

In input.txt there are names of sites for example: kosice.sk bratislava.sk presov.sk every name of the site is on a new line and they are in aplhabet order. And they are also as names of dir of the first level.

  • If I understood your question right, you can first search files that you need for something, and store them into a temp file. Then you can use xargs, something like cat list_of_files.txt | xargs -I % sh -c 'echo command1 %;'. Ofc this can be done in more elegant way. – Lev Bystritskiy Mar 14 '16 at 4:58
  • 2
    find /home/student/eny/abc -exec lynx.sh {} \; is just fine except you should add -type f to find only files not dirs. Not clear what you think is wrong/does not work? – Tagwint Mar 14 '16 at 8:37
  • You should add a parameter to -iname and you should try to used execplus (-exec ... {} +) if there are many files to work on. – schily Mar 14 '16 at 13:13
  • @LevBystritskiy I can not store files in a temp file because it is too many files. As I mentioned before I have downloaded about 3200 webpages to the depth 2. So ist lots of files. I have to work with all files from all webpages. – eny Mar 31 '16 at 21:51
  • @Tagwint It does not work for all files in all folders. It only edits maybe first folder to the depth 1 and other files and other folders (from 3200 amount) and are untouched. – eny Mar 31 '16 at 21:53
0

Here is the final version of your command

find /home/student/eny/abc -type f -exec ./lynx.sh {} \;

Points to note:

  1. -type f finds files only
  2. you should specify path to your script ./ (dot slash) means current dir, you may want to specify the full path
  3. lynx.sh should have executable bit set file mode 0755 would be fine
  • Lynx.sh is in the current dir so probably path will not be needed (I hope so). Well, this is the first time it does somenthing different than usually. This is what it does after your new command: ./lynx.sh: line 17: /bla_lynx.txt: Permission denied #!/bin/bash ./lynx.sh: line 17: /bla_lynx.txt: Permission denied fileA=$1 ./lynx.sh: line 17: /bla_lynx.txt: Permission denied while IFS= read -r lineA ./lynx.sh: line 17: /bla_lynx.txt: Permission denied do Actually it re-writes everything in my lynx.sh and started re-writing everything else what find has found. – eny Apr 1 '16 at 16:26
  • lynx.sh expects 2 parameters, first find command provides only one, so the $2 placeholder in your script is empty which results in attempt to write to /blalynx.txt file. That is aboslute path where non-root user does not have permition to write. I did not revise the correctness of the script, assuming it's fine and you only need help with find command. I suggest you first test your script w/o find iterations, start it manually with an arbitrary file that find would find. Once being sure it works with one file, apply your find commands. – Tagwint Apr 1 '16 at 19:28
  • I have tested this script before and it worked fine. Also before you helped me, some of my tries to write path worked in a way that it creates one file bla_lynx.sh correctly but not for all files, so I thought path is the problem. – eny Apr 1 '16 at 21:56
  • I have tested it once again on one file and it creates bla_lynx.sh and it was perfect, but when I add find, it stopped working and do what I said before. So on the same file it works without find and when I add find it shows "Permission denied" @Tagwint – eny Apr 2 '16 at 14:47
  • Please show the exact command you tested it so we can find out the differesnce from find approach – Tagwint Apr 2 '16 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.