Before anything else: be sure there no other devices like smartphones or laptops are connecting to the same USB external device.
This may vary according to OS, kernel and hardware, but I think it may be helpful, as I have found a lot of unsolved issues posted here.
A large amount of information is available at this page: Linux bluetooth setup with bluez and hcitool, on which this answer is partly based.
Connecting/pairing the device
First, be sure that your computer bluetooth is on, that the speaker is powered and its bluetooth capability activated: press the power button until a sound is heard, the same for the bluetooth button. At this point a small light has to be blinking on the bluetooth button.

Following the instruction from the link above, and assuming that all necessary tools were already installed, I have tried to start scanning with the command sudo hciconfig hci0 up
. But that gave Can't init device hci0: Operation not possible due to RF-kill (132)
.
Going back to the instructions
Before start scanning make sure that your bluetooth device is turned
on and not blocked, you can check that with the rfkill command
So:
sudo rfkill list
...
6: hci0: Bluetooth
Soft blocked: yes
Hard blocked: no
The computer's bluetooth was 'soft blocked' for some reason. That seemed to be the cause of the initial problem: the bluetooth speaker not being seen by the computer.
So, back to instructions:
If the bluetooth device is blocked (soft or hard blocked), unblock it
with the rfkill command again
sudo rfkill unblock bluetooth
If that doesn't work, do
systemctl enable bluetooth.service
systemctl start bluetooth.service
as said here.
After that, running again
sudo hciconfig hci0 up
and then
hcitool scan
the bluetooth speaker should appear in the list and ready to be connected:

Switching between bluetooth and other output devices
At this point the bluetooth speaker is visible in the sound settings too (pavucontrol
), but the sound is still coming only from the previous output device (laptop speakers, wired speakers/headphones), namely the one that has the "Set as fallback" option checked under the Output Devices tab.

Changing that will not affect the already playing applications, but only those started after this change. Also, disconnecting and reconnecting the bluetooth device will reset this setting back to the internal/wired output (which then will be used by all newly started applications excepting the ones to which a specific device is allocated as said below).
In order to change the output device for an already playing application and in order to set an output device per application that will ignore the previously mentioned setting ('Set as fallback'), go to the Playback tab and chose for each playing application the output device:

Note that in this way specific devices can be allocated to different applications.
There is also a third party panel indicator: indicator-sound-switcher.

Considering interference with pavucontrol
: each setting overrides the setting of the other tool. The setting in indicator-sound-switcher affects all applications (the output device for all running applications listed under the pavucontrol
Playing tab, and the "Set to fallback" option under Output Devices tab).
In case reconnecting the bluetooth speaker will not work: see this question, the workaround therein, hopefully some good answer to it..
The UE BOOM2 device can be reset with a long push at the same time on the power button and the down volume button (until a sound is heard and the device shuts down), which may be helpful in combination with the above.