Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

In real world I have a xsd in a jar file across various releases I am trying to check whether xsd has changed across releases 10.x.y.z to 11.a.b.c

I have different release directories which are read only like below in some /m/n/i/10.x.y.z or 11.x.y.z. I want to search for abc.jar which can be buried in my release directory 10.x.y.z or 11.x.y.z in some p/q/r/abc.jar across all those releases and extract xyz.xsd from them and do a diff between them?Also i do not have write permission to the release directories.

How can I achieve it in Unix or shell script? I'm new to Unix. Directory structure

.
|-- 10.1.2.2.0
|   `-- GENERIC
|       `-- RELEASE
          --  x/y/z/abc.jar
|-- 10.1.2.3.0
|   `-- GENERIC
|       `-- RELEASE
          -- x/y/z/abc.jar
|-- 10.1.3.1.0
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.3.0
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.3.0-HOTPLUG
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.3.0BPA
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.3.0WEBSPHERE
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.3.1
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.4.0
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.4.0MLR1
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.4.0MLR10
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.4.0MLR2
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.4.0MLR3
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.4.0MLR4
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.4.0MLR5
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.4.0MLR6
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.4.0MLR7
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.4.0MLR8
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.4.0MLR9
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.5.0
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.5.0.QA.06012009
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.5.1
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.5.2
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3.5.3
|   `-- GENERIC
|       `-- RELEASE
|-- 10.1.3QAMLR6
|   `-- GENERIC
|       `-- RELEASE
|-- 11.1.1.1.0
|   `-- GENERIC
|       `-- RELEASE
|-- 11.1.1.1.0.BPA
|   `-- GENERIC
|       `-- RELEASE
|-- 11.1.1.1.0.CEP
|   `-- GENERIC
|       `-- RELEASE
|-- 11.1.1.2.0
|   `-- GENERIC
|       `-- RELEASE
|-- 11.1.1.3.0
|   `-- GENERIC
|       `-- RELEASE
`-- 11.1.1.4.0
    `-- GENERIC
        `-- RELEASE

93 directories, 0 files
share|improve this question
    
I updated my question Thanks –  constantlearner Apr 25 '13 at 19:05
1  
Please provide more details about the tree structure of the jar file. Also tell us please, how do you want to diff ? I assume there are more than two *.xsd files. –  Rany Albeg Wein Apr 25 '13 at 19:31
    
I have this directory structures in some /x/y/z/10.x.y.z.I have an xsd file xyz.xsd which in present in abc.jar across various releases.I want to check whether xsd files has been changed across various releases.The abc.jar can be anywhere inside my 10.x.y.z.Also i dont have execute permission i have read permission –  constantlearner Apr 25 '13 at 19:33
    
my abc.jar can be buried inside a/b/abc.jar across different releases –  constantlearner Apr 25 '13 at 19:55
    
What do you mean you have no exec permission? Do you mean you have no write permission? –  terdon Apr 25 '13 at 20:12

6 Answers 6

Assuming there is only one abc.jar under each directory:

basedir=path_to_your_Release_dirs
mkdir /var/tmp/reldiff
cd /var/tmp/reldiff
for x in Release1 Release2 Release3; do
   mkdir $x
   cd $x
   find $basedir/$x -name abc.jar -print0 | xargs -0 unzip -j xyz.xsd
   cd ..
done
diff3 */xyz.xsd
share|improve this answer
    
I have many directories like xyz abc mno ...... pqr –  constantlearner Apr 25 '13 at 16:19
2  
Maybe you should update your question so it does not look like you only have 3 directories. –  Anthon Apr 25 '13 at 16:22
    
I have updated my question thanks –  constantlearner Apr 25 '13 at 19:08

Something like that, maybe?

#!/bin/sh

number_of_releases = 3    

for i in $(seq 1 $number_of_releases)
do
cd Release$i
jar xf abc.jar xyz.xsd
mv xyz.xsd ../xyz.xsd_$i
cd ..
done

for i in $(seq 1 $(($number_of_releases-1)))
do
diff xyz.xsd_$i xyz.xsd_$(($i+1)) > diff_Release$(($i))_to_Release$(($i + 1))
done

EDIT see @rany-albeg-wein comment

share|improve this answer
    
The second argument to jar xf is the path of the file you want to extract, relatively to the root of the jar. –  lgeorget Apr 25 '13 at 16:15
    
I have many directories like xyz abc mno ...... pqr –  constantlearner Apr 25 '13 at 16:18
    
if xyz.xsd is at the same place in all the jarfiles, it's easy to set the right path in the script. If it's not your case, you'll have to extract entirely the jarfile and then search for xyz.xsd with find for example. –  lgeorget Apr 25 '13 at 16:23
1  
for i in {1..$n} is wrong. The Bash parser performs Brace Expansion before any other expansions or substitutions. So the brace expansion code sees the literal $n, which is not numeric, and therefore it doesn't expand the curly braces into a list of numbers. This makes it nearly impossible to use brace expansion to create lists whose size is only known at run-time. –  Rany Albeg Wein Apr 25 '13 at 18:23
    
oops yes you're right. Maybe seq could be of some help then. –  lgeorget Apr 25 '13 at 18:45

This script should do what you need. There are several assumptions it's making and it isn't overly modular. Doesn't have any checks either. Would need more specifics to make it more robust.

find . -type f -name 'abc.jar' -exec sh -c '
num=$(echo {} | sed 's#.*Release##' | sed 's#/.*##')
jar xvf {} xyz.xsd
mv xyz.xsd xyz.xsd.${num}
' {} \;

for i in xyz.xsd*; do 
  currnum=$(echo $i | sed 's#xyz.xsd.##')
  let nexnum=currnum+1
  [ ! -f xyz.xsd${nexnum} ] || exit
  echo "diff $i xyz.xsd${nexnum} > diffs_xyz.xsd_${currnum}_${nexnum}.txt"
done

Sample data

$ tree -A
.
├── myscript.bash
├── Release1
│   ├── abc.jar
│   └── xyz.jar
├── Release2
│   ├── abc.jar
│   └── xyz.jar
├── Release3
│   ├── abc.jar
│   └── xyz.jar
└── Release43
    └── xyz.jar

4 directories, 8 files
share|improve this answer
    
I have updated my question –  constantlearner Apr 25 '13 at 19:05

Here is a Perl solution:

#!/usr/bin/env perl
my $jar="abc.jar";       ## The jar file we will search for
my $target="xyz.xsd";        ## The file we will extract from it
my $data_dir=$ARGV[0]||"./";     ## The directory to search through

## Collect directory names, any directory that contains a file
## called whatever you set as $jar will be saved
my @dirs=split(/\n/,`find \"$data_dir\" -name \"$jar\" -exec dirname {} \\;`);
my @names;
foreach (@dirs) {
    ## Extract the $target from $jar
    system("cd \"$_/\"; jar xf \"$jar\" \"$target\"; cd -");
    ## Save the toplevel directory name
    my @a=split(/\//);
    push @names,$a[1];    
}
## Go through the list and run the diff
for ($i=0; $i<=$#dirs; $i++) {
    for ($k=$i+1; $k<=$#dirs; $k++) {
    system("diff \"$dirs[$i]/$target\" \"$dirs[$k]/$target\" > \"$names[$i]-$names[$k].$target.diff\"");
    }
}

If you save the Perl script as foo.pl and then run in in the directory that contains the various Release subdirectories, it will create these files:

Release1-Release2.xyz.diff
Release1-Release3.xyz.diff
Release3-Release2.xyz.diff

To run the script, either make it executable (chmod a+x foo) and run it directly (./foo.pl) or pass it to perl (perl foo.pl). The directory that contains the Release sub directories can be given as the 1st argument (if none is given, the current directory is assumed):

perl foo.pl /path/to/releases
share|improve this answer
    
I have many directories like xyz abc mno ...... pqr –  constantlearner Apr 25 '13 at 16:17
    
yes jar is buried inside aa/bb/abc.jar –  constantlearner Apr 25 '13 at 19:51
    
@constantlearner it should now work with nested jar files (and with directory names that contain spaces, just in case). –  terdon Apr 25 '13 at 20:08
    
thanks terdon why are you assigning $target="arachnophilia"; –  constantlearner Apr 25 '13 at 20:10
    
@constantlearner because that is what I was using to test on my machine, sorry, fixed now. –  terdon Apr 25 '13 at 20:13

You can use this function to extract all files from your jars

find /path/to/dir1 /path/to/dir2 -type f -name 'abc.jar' -exec bash -c 'jar xf "$1" "$2"' _ {} /path/to/file/inside/jar \;

As you can see, jar can take a second ( and on ) argument after xf which will be the name of the file you want to extract from the jar file. This path must be absolute within your jar tree structure. If you don't know the path to the file within the jar file, you can use this command, which will extract the whole jar file instead:

find /path/to/dir1 /path/to/dir2 -type f -name 'abc.jar' -exec bash -c 'jar xf "$1"' _ {} \;

If you used the first method then you can just iterate over your xsd files like this:

for i in *.xsd; do
    # diff your files here
done

If you used the second you'll have to use something like this:

while IFS= read -rd $'\0' xsd; do
    # diff your files here
done < <(find /path/to/relevant/location -type f -name '*.xsd' -print0)
share|improve this answer

A Git approach

Depending on needs it could be nice to use git as well. It could easily be extended to include other files etc. The script below is a simple variant of a more complex one I wrote for this purpose. Added find functionality.

It reads dates from extracted file and uses that in commit – thus you also get a nice time-view.


Sample usage

As a sample you could then diff on versions:

git diff

Or use GUI to nicely show differences. By conserving dates on committing the view also gives a good overview:

gitk --all &

Positioned in the git directory you could then also easily look at various versions by e.g.

 gvim abc.xsd
 git checkout 10.1.2.2.0
 # Look at this version
 # Go back
 git checkout master
 # View pretty log
 git log --date-order --graph --tags --simplify-by-decoration --pretty=format:'%ai %h %d'

And so on and so forth.


Quick walk-trough

  1. Create target directory and initialize Git. (Remove first if exists.)
  2. List folders in source directory and sort them by version sort. (Modify script by checking which sort gives best result.)
  3. Go trough each directory by order and locate source file.
  4. Copy file to target directory and extract needed file.
  5. Read modification date from file add and commit.

Script

#!/bin/bash

#
# Please read script thoroughly before use.
#

# Git directory (created, old deleted!)
git_dir="git_proj"

# Project directory
proj_root="$HOME/tmp/jardiff/sample_proj"
release_dir="GENERIC/RELEASE"
# If file always is in same location:
target_dir="$release_dir/x/y/z"

# Files to (find) copy and extract
jar_file="abc.jar"
xsd_file="abc.xsd"

# If we should use relative absolute target directory or find
use_find=1
# If we want to commit empty (no change, but to list version as separate commit)
commit_empty=0

# Remove existing and create project diff directory
[ -e "$git_dir" ] && (echo y | rm -r "$git_dir")
mkdir -p "$git_dir"

cd "$git_dir"

# Init empty git
git init

# Date is read from file
export GIT_COMMITTER_DATE
export GIT_AUTHOR_DATE
# Author unknown (or extract it from elsewhere)
export GIT_AUTHOR_NAME="Unknown"
export GIT_AUTHOR_EMAIL="john@example.net"
# Committer
export GIT_COMMITTER_NAME="Auto Batch"
export GIT_COMMITTER_EMAIL="batch@mydomain.com"


for dir in $(ls -d "$proj_root"/* | sort -V); do
    echo "Processing: $dir"
    # Use root directory as project version
    bn=$(basename "$dir")
    # Copy target to git directory
    if ((use_find)); then
        if ! fn=$(find "$dir/$release_dir" -name "$jar_file" -print0); then
            fn="" # Probably not needed
        fi
    else
        fn="$dir/$target_dir/$jar_file"
    fi
    if [ -e "$fn" ]; then
        printf "Copying in %s\n" "$fn"
        cp --preserve=timestamps "$fn" .
        # Extract
        jar xf "$jar_file" "$xsd_file"
        # Read modified timestamp
        ts=$(stat -c %y "$xsd_file")
        # Set Git env date variables
        GIT_COMMITTER_DATE="$ts"
        GIT_AUTHOR_DATE="$ts"
        # Add file and commit
        git add "$xsd_file"
    else
        # Use timestamp from root directory
        # Here you would probably use a README file
        # or something else that has a more likely correct date
        # either by a new find or relative path.
        ts=$(stat -c %y "$dir")
        printf "Tagging empty %s - %s\n" "$bn" "$ts"
        # Set Git env date variables
        GIT_COMMITTER_DATE="$ts"
        GIT_AUTHOR_DATE="$ts"
    fi

    if ((commit_empty)); then
        git commit --allow-empty -am "Release $bn"
    else
        git commit -am "Release $bn"
    fi
    # Tag as release
    git tag -a "$bn" -m "Release $bn"
done
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.