Pairwise combinations of filenames
If I have n files in a directory, for example;
a
b
c
How do I get pairwise combinations of these files (non-directional) to pass to a function?
The expected output is
a-b
a-c
b-c
so that it can be passed to a function like
fn -file1 a -file2 b
fn -file1 a -file2 c
...
This is what I am trying out now.
for i in *.txt
do
for j in *.txt
do
if [ "$i" != "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
Output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and a.txt
Pairs b.txt and c.txt
Pairs c.txt and a.txt
Pairs c.txt and b.txt
I still have duplicates (a-b is same as b-a) and I am thinking perhaps there is a better way to do this.
bash
add a comment |
If I have n files in a directory, for example;
a
b
c
How do I get pairwise combinations of these files (non-directional) to pass to a function?
The expected output is
a-b
a-c
b-c
so that it can be passed to a function like
fn -file1 a -file2 b
fn -file1 a -file2 c
...
This is what I am trying out now.
for i in *.txt
do
for j in *.txt
do
if [ "$i" != "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
Output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and a.txt
Pairs b.txt and c.txt
Pairs c.txt and a.txt
Pairs c.txt and b.txt
I still have duplicates (a-b is same as b-a) and I am thinking perhaps there is a better way to do this.
bash
Tryxargs -n.
– ctrl-alt-delor
Dec 23 '18 at 19:14
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 '18 at 19:38
add a comment |
If I have n files in a directory, for example;
a
b
c
How do I get pairwise combinations of these files (non-directional) to pass to a function?
The expected output is
a-b
a-c
b-c
so that it can be passed to a function like
fn -file1 a -file2 b
fn -file1 a -file2 c
...
This is what I am trying out now.
for i in *.txt
do
for j in *.txt
do
if [ "$i" != "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
Output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and a.txt
Pairs b.txt and c.txt
Pairs c.txt and a.txt
Pairs c.txt and b.txt
I still have duplicates (a-b is same as b-a) and I am thinking perhaps there is a better way to do this.
bash
If I have n files in a directory, for example;
a
b
c
How do I get pairwise combinations of these files (non-directional) to pass to a function?
The expected output is
a-b
a-c
b-c
so that it can be passed to a function like
fn -file1 a -file2 b
fn -file1 a -file2 c
...
This is what I am trying out now.
for i in *.txt
do
for j in *.txt
do
if [ "$i" != "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
Output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and a.txt
Pairs b.txt and c.txt
Pairs c.txt and a.txt
Pairs c.txt and b.txt
I still have duplicates (a-b is same as b-a) and I am thinking perhaps there is a better way to do this.
bash
bash
edited Dec 23 '18 at 22:30
rmf
asked Dec 23 '18 at 18:59
rmfrmf
292212
292212
Tryxargs -n.
– ctrl-alt-delor
Dec 23 '18 at 19:14
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 '18 at 19:38
add a comment |
Tryxargs -n.
– ctrl-alt-delor
Dec 23 '18 at 19:14
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 '18 at 19:38
Try
xargs -n.– ctrl-alt-delor
Dec 23 '18 at 19:14
Try
xargs -n.– ctrl-alt-delor
Dec 23 '18 at 19:14
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 '18 at 19:38
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 '18 at 19:38
add a comment |
4 Answers
4
active
oldest
votes
Put the file names in an array and run through it manually with two loops.
You get each pairing only once if if j < i where i and j are the indexes used in the outer and the inner loop, respectively.
$ touch a b c d
$ f=(*)
$ for ((i = 0; i < ${#f[@]}; i++)); do
for ((j = i + 1; j < ${#f[@]}; j++)); do
echo "${f[i]} - ${f[j]}";
done;
done
a - b
a - c
a - d
b - c
b - d
c - d
1
Note that it is better to useprintfrather thanecho: unix.stackexchange.com/questions/65803/…
– cryptarch
Dec 23 '18 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call tofn, instead ofechoorprintf.echoworks fine as an example here, though.
– ilkkachu
Dec 23 '18 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 '18 at 21:45
add a comment |
You're very close in your script, but you want to remove duplicates; i.e a-b is considered a duplicate of b-a.
We can use an inequality to handle this; only display the filename if the first file comes before the second file alphabetically. This will ensure only one of each matches.
for i in *.txt
do
for j in *.txt
do
if [ "$i" < "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
This gives the output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and c.txt
This isn't an efficient algorithm (it's O(n^2)) but may be good enough for your needs.
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 '18 at 20:08
Yes, but without knowing the cost offnit's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call tofntakes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.
– Stephen Harris
Dec 23 '18 at 20:16
add a comment |
With join trick for filenames without whitespace(s):
Sample list of files:
$ ls *.json | head -4
1.json
2.json
comp.json
conf.json
$ join -j9999 -o1.1,2.1 <(ls *.json | head -4) <(ls *.json | head -4) | awk '$1 != $2'
1.json 2.json
1.json comp.json
1.json conf.json
2.json 1.json
2.json comp.json
2.json conf.json
comp.json 1.json
comp.json 2.json
comp.json conf.json
conf.json 1.json
conf.json 2.json
conf.json comp.json
-joption points to a common field position to join on; but-j9999will provoke mixed joining resembling cartesian product.
add a comment |
for i in *.txt ; do
for j in *.txt ; do
if [ "$i" '<' "$j" ] ; then
echo "Pairs $i and $j"
fi
done
done
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f490649%2fpairwise-combinations-of-filenames%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
Put the file names in an array and run through it manually with two loops.
You get each pairing only once if if j < i where i and j are the indexes used in the outer and the inner loop, respectively.
$ touch a b c d
$ f=(*)
$ for ((i = 0; i < ${#f[@]}; i++)); do
for ((j = i + 1; j < ${#f[@]}; j++)); do
echo "${f[i]} - ${f[j]}";
done;
done
a - b
a - c
a - d
b - c
b - d
c - d
1
Note that it is better to useprintfrather thanecho: unix.stackexchange.com/questions/65803/…
– cryptarch
Dec 23 '18 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call tofn, instead ofechoorprintf.echoworks fine as an example here, though.
– ilkkachu
Dec 23 '18 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 '18 at 21:45
add a comment |
Put the file names in an array and run through it manually with two loops.
You get each pairing only once if if j < i where i and j are the indexes used in the outer and the inner loop, respectively.
$ touch a b c d
$ f=(*)
$ for ((i = 0; i < ${#f[@]}; i++)); do
for ((j = i + 1; j < ${#f[@]}; j++)); do
echo "${f[i]} - ${f[j]}";
done;
done
a - b
a - c
a - d
b - c
b - d
c - d
1
Note that it is better to useprintfrather thanecho: unix.stackexchange.com/questions/65803/…
– cryptarch
Dec 23 '18 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call tofn, instead ofechoorprintf.echoworks fine as an example here, though.
– ilkkachu
Dec 23 '18 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 '18 at 21:45
add a comment |
Put the file names in an array and run through it manually with two loops.
You get each pairing only once if if j < i where i and j are the indexes used in the outer and the inner loop, respectively.
$ touch a b c d
$ f=(*)
$ for ((i = 0; i < ${#f[@]}; i++)); do
for ((j = i + 1; j < ${#f[@]}; j++)); do
echo "${f[i]} - ${f[j]}";
done;
done
a - b
a - c
a - d
b - c
b - d
c - d
Put the file names in an array and run through it manually with two loops.
You get each pairing only once if if j < i where i and j are the indexes used in the outer and the inner loop, respectively.
$ touch a b c d
$ f=(*)
$ for ((i = 0; i < ${#f[@]}; i++)); do
for ((j = i + 1; j < ${#f[@]}; j++)); do
echo "${f[i]} - ${f[j]}";
done;
done
a - b
a - c
a - d
b - c
b - d
c - d
answered Dec 23 '18 at 19:56
ilkkachuilkkachu
58.2k890164
58.2k890164
1
Note that it is better to useprintfrather thanecho: unix.stackexchange.com/questions/65803/…
– cryptarch
Dec 23 '18 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call tofn, instead ofechoorprintf.echoworks fine as an example here, though.
– ilkkachu
Dec 23 '18 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 '18 at 21:45
add a comment |
1
Note that it is better to useprintfrather thanecho: unix.stackexchange.com/questions/65803/…
– cryptarch
Dec 23 '18 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call tofn, instead ofechoorprintf.echoworks fine as an example here, though.
– ilkkachu
Dec 23 '18 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 '18 at 21:45
1
1
Note that it is better to use
printf rather than echo: unix.stackexchange.com/questions/65803/…– cryptarch
Dec 23 '18 at 20:06
Note that it is better to use
printf rather than echo: unix.stackexchange.com/questions/65803/…– cryptarch
Dec 23 '18 at 20:06
@cryptarch, to be in line with the question, the content of the loop should be a call to
fn, instead of echo or printf. echo works fine as an example here, though.– ilkkachu
Dec 23 '18 at 21:24
@cryptarch, to be in line with the question, the content of the loop should be a call to
fn, instead of echo or printf. echo works fine as an example here, though.– ilkkachu
Dec 23 '18 at 21:24
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 '18 at 21:45
Sure, it's not broken, you already got my +1 ;)
– cryptarch
Dec 23 '18 at 21:45
add a comment |
You're very close in your script, but you want to remove duplicates; i.e a-b is considered a duplicate of b-a.
We can use an inequality to handle this; only display the filename if the first file comes before the second file alphabetically. This will ensure only one of each matches.
for i in *.txt
do
for j in *.txt
do
if [ "$i" < "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
This gives the output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and c.txt
This isn't an efficient algorithm (it's O(n^2)) but may be good enough for your needs.
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 '18 at 20:08
Yes, but without knowing the cost offnit's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call tofntakes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.
– Stephen Harris
Dec 23 '18 at 20:16
add a comment |
You're very close in your script, but you want to remove duplicates; i.e a-b is considered a duplicate of b-a.
We can use an inequality to handle this; only display the filename if the first file comes before the second file alphabetically. This will ensure only one of each matches.
for i in *.txt
do
for j in *.txt
do
if [ "$i" < "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
This gives the output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and c.txt
This isn't an efficient algorithm (it's O(n^2)) but may be good enough for your needs.
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 '18 at 20:08
Yes, but without knowing the cost offnit's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call tofntakes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.
– Stephen Harris
Dec 23 '18 at 20:16
add a comment |
You're very close in your script, but you want to remove duplicates; i.e a-b is considered a duplicate of b-a.
We can use an inequality to handle this; only display the filename if the first file comes before the second file alphabetically. This will ensure only one of each matches.
for i in *.txt
do
for j in *.txt
do
if [ "$i" < "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
This gives the output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and c.txt
This isn't an efficient algorithm (it's O(n^2)) but may be good enough for your needs.
You're very close in your script, but you want to remove duplicates; i.e a-b is considered a duplicate of b-a.
We can use an inequality to handle this; only display the filename if the first file comes before the second file alphabetically. This will ensure only one of each matches.
for i in *.txt
do
for j in *.txt
do
if [ "$i" < "$j" ]
then
echo "Pairs $i and $j"
fi
done
done
This gives the output
Pairs a.txt and b.txt
Pairs a.txt and c.txt
Pairs b.txt and c.txt
This isn't an efficient algorithm (it's O(n^2)) but may be good enough for your needs.
answered Dec 23 '18 at 20:05
Stephen HarrisStephen Harris
25.8k24477
25.8k24477
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 '18 at 20:08
Yes, but without knowing the cost offnit's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call tofntakes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.
– Stephen Harris
Dec 23 '18 at 20:16
add a comment |
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 '18 at 20:08
Yes, but without knowing the cost offnit's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call tofntakes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.
– Stephen Harris
Dec 23 '18 at 20:16
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 '18 at 20:08
This will take more than twice as long as unix.stackexchange.com/a/490657/305714 because you are checking each pair twice rather than restricting the loop to avoid redundancy
– cryptarch
Dec 23 '18 at 20:08
Yes, but without knowing the cost of
fn it's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call to fn takes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.– Stephen Harris
Dec 23 '18 at 20:16
Yes, but without knowing the cost of
fn it's hard to know if this overhead is significant or not. Taking 0.2s instead of 0.1s doesn't mean anything if every call to fn takes 1 second. Sometimes the naive algorithms are just fine ;-) In this case I just fixed the original code, rather than providing a more optimised alternative, because I considered it a better "teaching" solution.– Stephen Harris
Dec 23 '18 at 20:16
add a comment |
With join trick for filenames without whitespace(s):
Sample list of files:
$ ls *.json | head -4
1.json
2.json
comp.json
conf.json
$ join -j9999 -o1.1,2.1 <(ls *.json | head -4) <(ls *.json | head -4) | awk '$1 != $2'
1.json 2.json
1.json comp.json
1.json conf.json
2.json 1.json
2.json comp.json
2.json conf.json
comp.json 1.json
comp.json 2.json
comp.json conf.json
conf.json 1.json
conf.json 2.json
conf.json comp.json
-joption points to a common field position to join on; but-j9999will provoke mixed joining resembling cartesian product.
add a comment |
With join trick for filenames without whitespace(s):
Sample list of files:
$ ls *.json | head -4
1.json
2.json
comp.json
conf.json
$ join -j9999 -o1.1,2.1 <(ls *.json | head -4) <(ls *.json | head -4) | awk '$1 != $2'
1.json 2.json
1.json comp.json
1.json conf.json
2.json 1.json
2.json comp.json
2.json conf.json
comp.json 1.json
comp.json 2.json
comp.json conf.json
conf.json 1.json
conf.json 2.json
conf.json comp.json
-joption points to a common field position to join on; but-j9999will provoke mixed joining resembling cartesian product.
add a comment |
With join trick for filenames without whitespace(s):
Sample list of files:
$ ls *.json | head -4
1.json
2.json
comp.json
conf.json
$ join -j9999 -o1.1,2.1 <(ls *.json | head -4) <(ls *.json | head -4) | awk '$1 != $2'
1.json 2.json
1.json comp.json
1.json conf.json
2.json 1.json
2.json comp.json
2.json conf.json
comp.json 1.json
comp.json 2.json
comp.json conf.json
conf.json 1.json
conf.json 2.json
conf.json comp.json
-joption points to a common field position to join on; but-j9999will provoke mixed joining resembling cartesian product.
With join trick for filenames without whitespace(s):
Sample list of files:
$ ls *.json | head -4
1.json
2.json
comp.json
conf.json
$ join -j9999 -o1.1,2.1 <(ls *.json | head -4) <(ls *.json | head -4) | awk '$1 != $2'
1.json 2.json
1.json comp.json
1.json conf.json
2.json 1.json
2.json comp.json
2.json conf.json
comp.json 1.json
comp.json 2.json
comp.json conf.json
conf.json 1.json
conf.json 2.json
conf.json comp.json
-joption points to a common field position to join on; but-j9999will provoke mixed joining resembling cartesian product.
answered Dec 23 '18 at 19:36
RomanPerekhrestRomanPerekhrest
23k12447
23k12447
add a comment |
add a comment |
for i in *.txt ; do
for j in *.txt ; do
if [ "$i" '<' "$j" ] ; then
echo "Pairs $i and $j"
fi
done
done
add a comment |
for i in *.txt ; do
for j in *.txt ; do
if [ "$i" '<' "$j" ] ; then
echo "Pairs $i and $j"
fi
done
done
add a comment |
for i in *.txt ; do
for j in *.txt ; do
if [ "$i" '<' "$j" ] ; then
echo "Pairs $i and $j"
fi
done
done
for i in *.txt ; do
for j in *.txt ; do
if [ "$i" '<' "$j" ] ; then
echo "Pairs $i and $j"
fi
done
done
answered Dec 25 '18 at 21:43
Ole TangeOle Tange
12.4k1454105
12.4k1454105
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f490649%2fpairwise-combinations-of-filenames%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Try
xargs -n.– ctrl-alt-delor
Dec 23 '18 at 19:14
unix.stackexchange.com/q/11343/117549
– Jeff Schaller
Dec 23 '18 at 19:38