1

I have 2 files with the following data:

file 1

22:50:48] Return_M
22:50:49] Return_A
22:50:50] Return_F
22:50:51] Return_K
22:50:52] Return_Y

file 2

22:50:48] <0> X led_required
22:50:48] <0> X start_rules
22:50:48] <0> X leadstart
22:50:49] <0> X asynchronous_start
22:50:49] <0> X controldown
22:50:49] <0> X select_set(3)
22:50:49] <0> X limiting_rules
22:50:50] <0> X stock_manager
22:50:50] <0> X led_blink
22:50:50] <0> X start_required 
22:50:51] <0> X control_down
22:50:51] <0> X select_set(3)
22:50:51] <0> X start_rules
22:50:52] <0> X stock_manager
22:50:52] <0> X blink_led

I want to have a single file that outputs the following:

22:50:48] Return_M <0> X led_required
22:50:48] Return_M <0> X start_rules
22:50:48] Return_M <0> X leadstart
22:50:49] Return_A <0> X asynchronous_start
22:50:49] Return_A <0> X controldown
22:50:49] Return_A <0> X select_set(3)
22:50:49] Return_A <0> X limiting_rules
22:50:50] Return_F <0> X stock_manager
22:50:50] Return_F <0> X led_blink
22:50:50] Return_F <0> X start_required 
22:50:51] Return_K <0> X control_down
22:50:51] Return_K <0> X select_set(3)
22:50:51] Return_K <0> X start_rules
22:50:52] Return_Y <0> X stock_manager
22:50:52] Return_Y <0> X blink_led

It basically puts the return codes from file 1 into file 2 to but only at the matching time.

1

I've included two (slightly different) scripts that should do what you want: a Python script and a Bash script.

Python Solution

#!/usr/bin/env python3
# -*- encoding: utf-8 -*-
"""prefix_join.py"""

import sys

input_1 = sys.argv[1]
input_2 = sys.argv[2]

# Initialize a list to store prefixes as they occur
prefix_list = list()

# Parse the first input file
data_1 = dict()
with open(input_1, "r") as file_1:
    for line in file_1:

        # Remove trailing and leading whitespace
        line = line.strip()

        # Split the string on the first whitespace character
        prefix, sep, suffix = line.partition(" ")

        # Add the prefix the list of prefixes
        if prefix not in prefix_list:
            prefix_list.append(prefix)

        # Add the prefix to the first data dictionary
        if prefix not in data_1:
            data_1[prefix] = list()

        # Add the suffix to the data dictionary
        data_1[prefix].append(suffix)

# Parse the second input file
data_2 = dict()
with open(input_2, "r") as file_2:
    for line in file_2:

        # Remove trailing and leading whitespace
        line = line.strip()

        # Split the string on the first whitespace character
        prefix, sep, suffix = line.partition(" ")

        # Add the prefix the list of prefixes
        if prefix not in prefix_list:
            prefix_list.append(prefix)

        # Add the prefix to the first data dictionary
        if prefix not in data_2:
            data_2[prefix] = list()

        # Add the suffix to the data dictionary
        data_2[prefix].append(suffix)

# Output the joined data
for prefix in prefix_list:
    for value_1 in data_1.get(prefix, list()):
        for value_2 in data_2.get(prefix, list()):
            output_line = "{} {} X {}".format(prefix, value_1, value_2)
            print(output_line)

You would run it like this:

python prefix_join.py file-1.txt file-2.txt

On your example data, it produces the following output:

22:50:48] Return_M X <0> X led_required
22:50:48] Return_M X <0> X start_rules
22:50:48] Return_M X <0> X leadstart
22:50:49] Return_A X <0> X asynchronous_start
22:50:49] Return_A X <0> X controldown
22:50:49] Return_A X <0> X select_set(3)
22:50:49] Return_A X <0> X limiting_rules
22:50:50] Return_F X <0> X stock_manager
22:50:50] Return_F X <0> X led_blink
22:50:50] Return_F X <0> X start_required
22:50:51] Return_K X <0> X control_down
22:50:51] Return_K X <0> X select_set(3)
22:50:51] Return_K X <0> X start_rules
22:50:52] Return_Y X <0> X stock_manager
22:50:52] Return_Y X <0> X blink_led

Bash Solution

#!/usr/bin/env bash
# prefix-join.sh

# Get the input files as command-line arguments
input_file_1="$1"
input_file_2="$2"

# Set the internal field separator to be a newline (don't include spaces or tabs)
IFS=$'\n'

# Iterate over the lines of the first input file
for line_1 in $(cat "${input_file_1}"); do

    # Split the line on the first space
    prefix_1="${line_1/ */}";
    suffix_1="${line_1#*] }";

    # Iterate over the lines of the second input file
    for line_2 in $(cat "${input_file_2}"); do

        # Split the line on the first space
        prefix_2="${line_2/ */}";
        suffix_2="${line_2#*] }";

        # If the prefixes agree, combine the suffixes and output the result
        if [[ "${prefix_1}" = "${prefix_2}" ]]; then
            echo "${prefix_1} ${suffix_1} X ${suffix_2}"
        fi 

    done
done

You would run it like this:

bash prefix-join.sh file-1.txt file-2.txt

On your sample data, this produces the same output as the Python script.

  • Thank you for that, but is there anyway we can do that in shell? – Vineeth Thomas Mar 21 at 13:47
  • @VineethThomas What version of Bash (assuming you're using Bash) are you using? – igal Mar 21 at 14:25
  • @VineethThomas I added a (slightly different, less robust) Bash solution. – igal Mar 21 at 14:56
0

I Done by below command

command

for ((i=1;i<=5;i++)); do j=`awk -v i="$i" 'NR==i{print $1}' g1.txt`;k=`awk -v i="$i" 'NR==i{print $NF}' g1.txt`;awk -v j="$j" -v k="$k" '$1==j{print $1,k,$2,$3,$4}' g2.txt;done

output

for ((i=1;i<=5;i++)); do j=`awk -v i="$i" 'NR==i{print $1}' g1.txt`;k=`awk -v i="$i" 'NR==i{print $NF}' g1.txt`;awk -v j="$j" -v k="$k" '$1==j{print $1,k,$2,$3,$4}' g2.txt;done

22:50:48] Return_M <0> X led_required
22:50:48] Return_M <0> X start_rules
22:50:48] Return_M <0> X leadstart
22:50:49] Return_A <0> X asynchronous_start
22:50:49] Return_A <0> X controldown
22:50:49] Return_A <0> X select_set(3)
22:50:49] Return_A <0> X limiting_rules
22:50:50] Return_F <0> X stock_manager
22:50:50] Return_F <0> X led_blink
22:50:50] Return_F <0> X start_required
22:50:51] Return_K <0> X control_down
22:50:51] Return_K <0> X select_set(3)
22:50:51] Return_K <0> X start_rules
22:50:52] Return_Y <0> X stock_manager
22:50:52] Return_Y <0> X blink_led
0

Note: This is a re-run of my previous answer to you (on a newer question, but I missed this question when it was fresh)

Assuming the two files are sorted, which they appear to be:

$ join file1 file2
22:50:48] Return_M <0> X led_required
22:50:48] Return_M <0> X start_rules
22:50:48] Return_M <0> X leadstart
22:50:49] Return_A <0> X asynchronous_start
22:50:49] Return_A <0> X controldown
22:50:49] Return_A <0> X select_set(3)
22:50:49] Return_A <0> X limiting_rules
22:50:50] Return_F <0> X stock_manager
22:50:50] Return_F <0> X led_blink
22:50:50] Return_F <0> X start_required
22:50:51] Return_K <0> X control_down
22:50:51] Return_K <0> X select_set(3)
22:50:51] Return_K <0> X start_rules
22:50:52] Return_Y <0> X stock_manager
22:50:52] Return_Y <0> X blink_led

The join utility performs a relational INNER JOIN operation on its two input files. By default, the first column is the join column, and the input needs to be sorted on this column.

The other thing, which is different for this question compared to the newer one, is to note the ordering of the columns. By default, join always puts the join column first, followed by all columns from the first file and then all columns from the second file.

You may change this order by using the -o flag with join (see manual).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.