We are using Cyclone V, which essentially is a SOC comprised of FPGA + ARM core. Is it possible to encrypt the root file system and decrypt it by using U-Boot with a key which is located in the FPGA?
You can do so. In this case you need to boot FPGA before ARM. Thus you can read the key from FPGA. Final realization depends on your preferences. It can be simply to read the key located in shared memory inside the chip. Or you can implement some communication protocol between FPGA and U-Boot. It is worth to consider whether you can decrypt rootfs in the FPGA. You will have hardware acceleration in this way.