This will be an Intel-centric answer.
Note that at the kernel level, the Spectre/Meltdown mitigations are not a single monolithic lump, but each Spectre/Meltdown variant has its own set of mitigations.
The kernel-level mitigation for Meltdown is called Page Table Isolation. The kernel can do it on its own, with a significant performance impact.
It helps if the processor+microcode offers Process Context IDentifiers (PCID), and even more if the INVPCID instruction is available in the processor: the Meltdown-fixing microcode updates added one or both of these capabilities, according to what the microcode could do on each particular processor model. Linux can certainly use these if they're available, and they can significantly reduce the performance impact of Meltdown mitigation.
Of course, the best situation is if the Page Table Isolation is not needed at all, and this is apparently what the "Hardware" fix for Meltdown means.
And if you think your system is secure enough and performance is more important to you, there is an option to turn the Meltdown fix off.
Spectre Variant 1
This is mitigated by patching the compiler to handle pointers more carefully in specific situations, and then rebuilding the kernel (and hypervisor, if any) and any security-critical binaries using the patched compiler. This has been done. It adds some extra instructions to the key locations in the compiled code, which makes programs slightly larger and slower to execute, but fortunately the difference is usually tiny.
As far as I understand, the microcode and the processor design cannot do much here: even Cascade Lake indicates only OS/VMM-level fixes for this.
Spectre Variant 2
This is the tricky one, and probably hardest to fix, as a true fix requires some innovation at the level of CPU design theory.
Even without any assistance from the CPU hardware+microcode, the kernel can implement retpoline, i.e. RETurn instruction trampoline. That is an Assembler-level programming technique. As with the Meltdown mitigation, since implementing this takes a non-trivial chunk out of system performance, you have the option to disable this mitigation of you need performance more than security.
The CPU+microcode can again provide some new features that make the Spectre V2 mitigation easier for the kernel:
- Indirect Branch Restricted Speculation (IBRS), basically a switchable mode in the CPU's branch predictor that prevents user-space code from influencing the branch prediction in the kernel-space across the user->kernel mode transition.
- Indirect Branch Predictor Barrier (IBPB): basically a new CPU instruction that stops any code before it affecting the branch prediction in the code after the IBPB instruction.
- Single Thread Indirect Branch Predictors (STIBP): a way to stop hyper-threads on the same core from sharing branch prediction information.
The Linux kernel can use these features where appropriate, to help minimize the performance loss caused by the retpoline technique.
See this Intel PDF for more details.