Another option is to use a language that supports
xor. e.g. with perl:
$ mkdir dir
$ touch dir/filea
$ perl -le 'print (((-f $ARGV) xor (-f $ARGV)) ? 0 : 1)' dir/filea dir/fileb
You can, of course, use command substitution to capture the output into a variable.
Or if you want the result as an exit code that you can use directly with
$ perl -le 'exit (((-f $ARGV) xor (-f $ARGV)) ? 0 : 1)' dir/filea dir/fileb
$ echo $?
Note: it's important to realise that perl's definition of true (non-zero/non-empty/defined) and false (0/empty string/undefined) is different to/the opposite of shell's definition of true (0) or false (non-zero). That's why the ternary operators above are written to return 0 when the xor evaluates as true and 1 when it evaluates as false.
That's not very important in a trivial script like this, but if you wanted to do more boolean calculations in the perl script then you'd need to keep working with perl's definition of true & false until the last moment when you return the true/false value shell expects.
BTW, if you think this is a bit odd, you're right. It is. It does make sense, though, and there's a good reason for it. But Shell is the odd one out. Most languages define true as non-zero and false as zero (and many have specific boolean data types, or only allow integers to be treated as booleans...with conversion required for other types. perl is very flexible in what it will interpret as true/false).
Anyway, Shell does it the other way around because it mostly/often needs to know if a program exited successfully (0) or with an error code (from 1 up to 127, depending on the program, each indicating a different error condition). It's not really true vs false, it's success vs error-code. Any other program which needs to test the exit code returned from an external program has to do the same, regardless of the language's own internal definition of true/false.
Note 2: it's not advisable to run perl (or any external program, really) repeatedly in a shell loop. The startup overhead might only be a few milliseconds on a fast, modern system, but that adds up with hundreds or thousands of loop iterations and makes it extremely, tediously slow. If you absolutely must do it in a shell loop, best to find another way. Otherwise, if you need to do the test in a loop and can't afford to fork external programs repeatedly, it's best to rewrite the whole script (or the time-sensitive parts of it) in perl or awk or python or anything-that-isn't-shell. See Why is using a shell loop to process text considered bad practice? for more on this topic.