Oh the joys.
This all boiled down to wether using
migrate commands with or without target nodes. From the current cli documentation via
crm resource help migrate:
If the destination node is left out, the resource is migrated by
creating a constraint which prevents it from running on the current
node. For this type of constraint to be created, the force argument is
In the past this could also be done without
force, and when you ran a two-node-cluster (two nodes never constitute a proper cluster but lets not digress here), you ended up with a statement in pacemakers running config telling it explicitly to not run on the node it ran on when the
migrate command was fired, so pacemaker would force it to move away onto another node.
If you ran a
crm resource migrate you ended up with a
location cli-... statement up in your pacemaker config. There are two types of such statements, which can be checked for rather easily:
crm configure show | grep -e cli-prefer -e cli-ban
For serious business, the command above should also be part of nagios/icinga/whatever monitoring check, as these manual constraints should never be present in running configuration in the long term anyway.
If you migrated again back to another node, you ended up with yet another such statement, marking the last node ALSO as a "non-runnable" location for the resource. If you had eventually all your nodes 'outmigrated' in such fashion... your resource in question would not start anywhere and you were out of luck, which is likely the reason this question was asked in the first place.
Solution was to ALWAYS have an
crm resource migrate command being followed by an
crm resource unmigrate command when omitting target nodes, to not shoot yourself in the foot eventually later.
If you however always specify a target node, all this does not matter:
Upon rerunning a
crm resource migrate RESOURCE NODE /
pcs resource move RESOURCE NODE the
cli-prefer... statement within the pacemaker config gets adjusted, so no worries.
So you can run
crm resource migrate /
pcs resource move as often as you like without running into trouble of resources not being able to start anymore.
To get rid of any of these manual constraints for a resource in particular, run one of these:
crm resource unmigrate RESOURCE
crm resource clear RESOURCE
pcs resource clear RESOURCE
This however will not make your resources magically to their intial hosts, if you did not create location constraints in the first place - unlike the other answer here suggests.