I was testing bache on raspberry pi 4 with ubuntu. The reason I choose ubuntu that I found standard raspbian got some issues with bcache as kernel module not properly loaded. I tried to troubleshoot bit but then I move to ubuntu and it works straight away
My setup is like this.
1 x 1TB HGST 5400RPM 2.5 laptop hard disk
1 x 256GB WD Green 2.5 SSD
Raspberry pi 4 4GB model with large heat-sink for cooling and 4A power.
I hooked up both HDD and SSD to the raspberry pi (both externally powered) using USB 3.0 ports and boot to ubuntu. First I tested the the under-voltage errors and found all normal.
SSD -> /dev/sda
HDD -> /dev/sdb
Then I create 1 partition on both drives and create the bcache as follows.
make-bcache -B /dev/sdb1
make-bcache -C /dev/sda1
then I mount the /dev/bcache0 on /datastore
then I attached the cache device as follows
echo MYUUID > /sys/block/bcache0/bcache/attach
Then I enabled write-back cache
echo writeback > /sys/block/bcache0/bcache/cache_mode
Then I installed vsftpd server and make the root ftp dir as my bcache0 mount point and I started testing. First few tests I can upload files 113MBps and I notices most of the files directly write in to the backing device even if the cache is attached.
when I tested the status using bcache-status script https://gist.github.com/damoxc/6267899 I saw most of the writes misses cache and directly writing to backing device and the 113MBps is directly from the mechanical hard drive :-O ?
Then I started to fine tune. As suggested on Troubleshooting performance part of this https://www.kernel.org/doc/Documentation/bcache.txt document
first I set sequential_cutoff to zero by executing this command
echo 0 > /sys/block/bcache0/bcache/sequential_cutoff
After this I can instantly see SSD device cache hits are increased. And at the same time I was running iostat continuously. And I was able to see from the iostat SSD is directly being accessed. But after few minutes my filezilla client hangs and I cannot restart the FTP upload stream. And when I try to access the bcache0 mount it's really slow. cache status was showing as "dirty"
Then I restart the pi and again attached the device. and set below stetting
echo 0 > /sys/fs/bcache/MYUUID/congested_read_threshold_us
echo 0 > /sys/fs/bcache/MYUUID/congested_write_threshold_us
According to https://www.kernel.org/doc/Documentation/bcache.txt article this is for avoid bcache track backing device latency. But even after this option. my FTP upload stream continuously crashing. Then I set all back to default. Still with large number of file uploads it crashes
And I noticed within the test pi CPU is not fully utilized.
The maximum throughput I can get using pi 4 1Gbps Ethernet is 930Mbps, which is extremely good. The HGST drive when I tested with crystal disk mark with NTFS able to write up to 90MBps. It seems I can get 113MBps on pi since the file system is ext4.
If I can get more than 80MBps ftp upload speed I'm ok with that. My questions are
Why FTP stream keep crashing when using with bcache and why bcache mount getting slow overtime.
why there is very low cache usage even with sequential_cutoff set to 0
has anyone tested bcache before with Raspberry PI 4 ? if yes how can I use the SSD for caching properly
And finally can someone explain more about how bcache actually works when It is on writeback mode. I only use this for archival data and I don't need access hot data on SSD kind of setup.
