(Warning: I haven't tested any of these commands. There may be typos or errors. Try everything with echo or ls and no sudo first.)
General advice: xargs is rarely the simplest way to do something. Without -i (which is deprecated by the way, use the portable -I {} instead), xargs expects an input format that no standard or common tool produces. With -I, xargs does have some use, though if you're piping find into it, find -exec is simpler.
You need a shell to expand {}/*, and that shell must act after {} has been replaced by one actual path. A simple solution is to do the line-by-line processing in the shell. Note the use of three patterns to capture all the files in a directory as * omits dot files; it doesn't matter if some of the patterns don't match any file, as rm -f ignores any argument that doesn't reference an existing file.
… |
while read -r line; do
sudo rm -rf -- "$line"/* "$line"/.[!.]* "$line"/..?*
done
Xargs is no use here since you have to process the files one by one anyway at some point, in order to get the shell to perform the globbing. As the general technique can be useful however, here's a way to feed file names back into a shell from xargs. The _ is the $0 argument to the shell (it's there in part not to have to bother to stick $0 back onto any subsequent argument, and to ensure that everything works even if $1 begins with a - and so would otherwise be treated as an option to the shell). There are two approaches here: either have the shell loop over its arguments, or tell xargs to pass a single argument to each shell invocation.
… | xargs -I {} sudo sh -c 'for x; do rm -rf -- "$x"/* "$x"/.[!.]* "$x"/..?*; done' _ {}
… | xargs -I {} -n 1 sudo sh -c 'rm -rf -- "$1"/* "$1"/.[!.]* "$1"/..?*' _ {}
I'm assuming that ls -d a* is a toy example and that you have a more complex command that produces one file name per line. Note that ls -d a* would not work there, as most ls implementations do not output nonprintable characters transparently. If the … is a find command, use -exec, as in
find a* -exec sh -c 'rm -rf -- "$1"/* "$1"/.[!.]* "$1"/..?*' _ {} \; -prune