2

I was given a problem and my solution passed the initial test cases but failed 50% on submit.

The problem: A directory contains a number of files and folders, some of these files are different types of logs; error.log, error.log.1, error.log.2, access.log.1, access.log.2 ..etc The contents of those files map to the following day, so 'cat error.log.1' has 'Day 2 logs' .. etc

The task is to increment the number at the end of the logs only, and leave the rest of the content of the directory unchanged. Also, create an empty file for each log type.

For example:

./
example_dir
example2_dir
error.log
error.log.1
info.log.20
access.log.1
readme.txt

Script changes the directory to:

./
example_dir (unchanged)
example2_dir (unchanged)
error.log (empty)
error.log.1 (originally error.log)
error.log.2 (originally error.log.1)
info.log (empty)
info.log.21 (originally info.log.20)
access.log (empty)
access.log.2 (originally access.log.1)
readme.txt (unchanged)

Conditions: # Files in the directory < 1000, Max #Files per type < 21

My solution:

#!/bin/bash

declare -a filenames

# Renaming in ascending order will lead to overwrite; so start rename from the bottom

files=$(find . -maxdepth 1 -name "*.log.*" -exec basename {} \; | sort -rn)


for i in $files; do

    currentFileNumber=$(echo -e "$i" | sed -e 's/[^0-9]*//g') # Extract the current number from the filename
    fileName=$(echo -e "$i" | sed -e 's/\.[0-9]*$//g') # Extract the name without the trailing number

    newFileNumber=$(("$currentFileNumber" + 1)) # Increment the current number

    mv "$i" "$fileName.$newFileNumber" # Rename and append the incremented value

    if [[ ! ${filenames[*]} =~ ${fileName} ]] # Store names of existing types to create empty files
    then
        filenames=("${filenames[@]}" "${fileName}")
    fi
    # Could make use of [[ -e "$fileName.log" ]] instead of an array, but won't pass the test for some reason
done

for j in "${filenames[@]}"; do touch "$j"; done # Create the empty files
unset filenames

It didn't show the test cases which I failed, so I am not really sure how to solve this any better.

  • 1
    Not an answer, but in the real world you don't write that yourself, but use something like logrotate for it. – dirkt Aug 7 at 10:16
  • I strongly agree @dirkt ; do not try to write it (but for fun). – francois P Aug 7 at 11:08
  • Sure, but this problem was meant to test bash scripting skills given to me in an interview. – x300n Aug 7 at 11:12
  • I can see 2 bugs in your script, the first being your script never renames error.log to error.log.1 as required by the example. If you are given an example input and output you should check your program works for it. If I was looking at your solution I would give it low marks due to the excessive number of processes being used. It is good that you have spotted you need to do the renames in reverse order. Why do you think they tell you that there are less than 21 files per file type? – icarus Aug 7 at 16:24
  • 1
    "Why there would be a second bug?" - because you wrote incorrect code! There are a number of ways to solve it, but for probably the most straight forward way (but not the best) the condition of 21 (or another smallish number) is important to keep the speed reasonable. I will post my solution tomorrow. In the mean time think about edge cases. You missed the edge case of going from no extension to having .1, which other edge cases did you miss? I am not saying there is only one more bug to find, just that I spotted 2 bugs in your code when looking at it briefly. – icarus Aug 8 at 0:22
2

@Haxiel has posted a solution. This is simular to the one I had in mind to the one I described as "most straight forward". I would have used a for loop than the until loop.

This is something that uses almost the minimal number of external processes, one mv for each existing file and one touch at the end to create the new files. ( The touch could be replaced by a loop creating the files using redirection to reduce the number of external processes by 1 ).

#!/bin/bash
shopt -s nullglob # Reduce the number of things we have to work with

# get a list of the files we want to work with. 
files=( *.log *.log.[1-9] *.log.[1-9][0-9] )

# reverse the list into rfiles, getting rid of non-file things
rfiles=()
for ((i=${#files[@]}-1;i>=0;i--)) ; do
        if [ -f "${files[i]}" ] ; then
                rfiles+=("${files[i]}")
        fi
done

# exit early if there is nothing to do
if [ ${#rfiles[@]} -eq 0 ] ; then
        exit 0
fi

# an array of the files we need to create
typeset -A newfiles

# Loop over the reversed file list
for f in "${rfiles[@]}"; do
    # Get everything up to the last "log"
    baseName=${f%log*}log
    # Remove up to the last "log" and then the optional "."
    currentFileNum=${f#"$baseName"}
    currentFileNum=${currentFileNum#.}
    mv -v "$f" "$baseName.$((currentFileNum+1))"
    # record the name to make the new files
    newfiles[$baseName]=1
done

# Create all the needed new files, using the names stored in the array
touch "${!newfiles[@]}"

The order this does things is different to to one produced by @Haxiel's solution, this moves first all the files with 2 digit numbers, then all the files with single digit numbers, and finally the files ending in ".log", rather than processing all the files with the same first parts together.

The original question said that there are less than 1,000 files and less than 21 versions per file. It didn't say what to do if there are more than this number. This solution caters for up to 100 versions per file, and can be extended to 1000 or more just by extending the pattern.

The number of files is limited by the amount of memory available to bash.

I believe this to be a better solution as it only tries to deal with files that exist rather than trying N files for every name. When N is small (like 21) this doesn't matter.

| improve this answer | |
  • Great! Thanks alot, I wasn't familiar with shopt and man page says it replaces unmatched patterns with null strings that is files that don't match the three patterns in the second line. But the patterns already match what is needed, so what's the point of that command? I like how you used associative arrays to process uniq basenames, very clever. – x300n Aug 9 at 7:08
  • 1
    The point of the shopt is for the case when the pattern doesn't match anything. So if for example there was only the file info.log.1 in the current directory, without the shopt then you would have files containing ".log", "info.log.1" and ".log.[1-9][0-9]". There is defense in depth here as we only add the entries in files to rfiles if it is a file, but if we only don't put ".log" and ".log.[1-9][0-9]" into files in the first place we don't need to worry about copying them to rfiles. – icarus Aug 9 at 7:18
2

This was a fun exercise, so here's my solution.

#/bin/bash
log_names=$(for logfile in $(find . -type f -name '*.log*'); do echo ${logfile%.[0-9]*}; done | sort -u)

for name in $log_names; do
    echo "Processing $name"
    i=20
    until [[ "$i" -eq 0 ]]; do
        if [[ -f "$name.$i" ]]; then
            next_num=$((i+1))
            mv -v "$name.$i" "$name.$next_num"
        fi
        i=$((i-1))
    done
    if [[ -f "$name" ]]; then
        mv -v "$name" "$name.1"
    fi
    touch "$name"
done

The log_names variable uses a find command to get the list of log files. Then, I apply a string substitution to remove the numeric suffix. After that, I sort and remove the duplicates.

At this point, I get a list of the unique log file names in the directory: ./access.log ./error.log ./info.log.

Then, I process each name in turn using a for loop.

Now, for each file, we have been told that the maximum possible number is 20. We start there, and use an until loop to count down.

The mv logic is simple: if 'filname.number' exists, move it to 'filename.(number+1)'.

When the until loop is finished (i = 0), we may have one unrotated file left - the one without the numeric suffix. If it does, move it to filename.1.

The last step is to create an empty file with touch.


Sample execution:

$ ls
access.log.1  error.log  error.log.1  example_dir  example2_dir  info.log.20  readme.txt  rotate.bash
    
$ bash rotate.bash
Processing ./access.log
'./access.log.1' -> './access.log.2'
Processing ./error.log
'./error.log.1' -> './error.log.2'
'./error.log' -> './error.log.1'
Processing ./info.log
'./info.log.20' -> './info.log.21'

$ ls -1
access.log
access.log.2
error.log
error.log.1
error.log.2
example_dir
example2_dir
info.log
info.log.21
readme.txt
rotate.bash
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.