2

I have a mini PC (Intel Celeron J4005, 4GB RAM, Intel Gigabit NIC), configured with:

  • Ubuntu (5.4.0-81-generic, installed on sda)
  • Samba (version 4.11.6-Ubuntu)
  • FTP (vsftpd, no encryption)
  • RAID5 (mdadm, md0: sdb-sdc-sdd, USB-SATA)

The RAID array is shared via Samba and FTP, but I want to eliminate FTP, all major clients are Windows machines. The problem is that I get way slower speeds through Samba share than FTP:

Device Method Read Speed (Mbyte/s, one large file)
md0 local ~220
md0 LAN, FTP ~115 (network limit)
md0 LAN, Samba ~48
md0 LAN, Samba, second run (cached in memory) ~115 (network limit)
sda LAN, Samba ~115 (network limit)

I tried with default Samba settings and with the current one (attached below), but I got the same result. I flushed the cache between tests.

iostat output sample (LAN, Samba, first run):

Device            r/s     rkB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wkB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dkB/s   drqm/s  %drqm d_await dareq-sz  aqu-sz  %util
md0            793.00 433408.00     0.00   0.00    0.00   546.54    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00   0.00
sda              0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00   0.00
sdb            254.00  16768.00     8.00   3.05   14.74    66.02    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    3.27  84.80
sdc            171.00  16896.00    93.00  35.23    2.99    98.81    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.32  60.80
sdd            161.00  16640.00   101.00  38.55   11.74   103.35    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    1.57  96.00

iostat output sample (LAN, FTP, first run):

Device            r/s     rkB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wkB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dkB/s   drqm/s  %drqm d_await dareq-sz  aqu-sz  %util
md0           1828.00 292480.00     0.00   0.00    0.00   160.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00   0.00
sda              0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00   0.00
sdb            458.00  39040.00   153.00  25.04    1.66    85.24    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.18  75.60
sdc            457.00  38976.00   152.00  24.96    1.45    85.29    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.09  70.40
sdd            457.00  38976.00   152.00  24.96    1.59    85.29    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.15  75.20

I have no clue what the problem can be, can someone help me, or at least where I should start investigating?


Samba config:

[global]
   workgroup = WORKGROUP
   min protocol = SMB3
   
   log level = 1
   socket options = TCP_NODELAY SO_RCVBUF=65536 SO_SNDBUF=65536 IPTOS_LOWDELAY SO_KEEPALIVE
   use sendfile = true
   aio read size = 65536
   aio write size = 65536
   read raw = yes
   write raw = yes
   getwd cache = yes
   acl allow execute always = true
   
   log file = /var/log/samba/log.%m
   max log size = 1000
   logging = file
   server role = standalone server
   obey pam restrictions = yes
   unix password sync = yes
   passwd program = /usr/bin/passwd %u
   passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
   pam password change = yes
   map to guest = bad user

[Share]
  path = /media/hdd
  writable = yes
  valid users = myuser
  directory mode = 0770
  create mode = 0660

RAID array configuration:

/dev/md0:
           Version : 1.2
     Creation Time : Tue Sep  7 13:19:26 2021
        Raid Level : raid5
        Array Size : 976441344 (931.21 GiB 999.88 GB)
     Used Dev Size : 488220672 (465.60 GiB 499.94 GB)
      Raid Devices : 3
     Total Devices : 3
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Tue Sep 9 14:37:52 2021
             State : clean
    Active Devices : 3
   Working Devices : 3
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 64K

Consistency Policy : bitmap

Filesystem info:

root@MiniPC:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            1.8G     0  1.8G   0% /dev
tmpfs           371M   12M  360M   3% /run
/dev/sda2        58G  3.4G   55G   6% /
tmpfs           1.9G   12K  1.9G   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
tmpfs           500M   79M  422M  16% /var/cache/apt
tmpfs           500M     0  500M   0% /tmp
tmpfs           500M     0  500M   0% /var/backups
tmpfs           500M  2.2M  498M   1% /var/log
tmpfs           500M     0  500M   0% /var/tmp
/dev/sda1       511M  5.3M  506M   2% /boot/efi
/dev/md0        917G  356G  562G  39% /media/hdd

root@MiniPC:~# lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda       8:0    0  59.6G  0 disk
├─sda1    8:1    0   512M  0 part  /boot/efi
└─sda2    8:2    0  59.1G  0 part  /
sdb       8:16   0 465.7G  0 disk
└─sdb1    8:17   0 465.7G  0 part
  └─md0   9:0    0 931.2G  0 raid5 /media/hdd
sdc       8:32   0 465.8G  0 disk
└─sdc1    8:33   0 465.8G  0 part
  └─md0   9:0    0 931.2G  0 raid5 /media/hdd
sdd       8:48   0 465.8G  0 disk
└─sdd1    8:49   0 465.8G  0 part
  └─md0   9:0    0 931.2G  0 raid5 /media/hdd
2
  • 1
    What filesystem did you use when formatting the RAID array? FUSE FS Drivers (like NTFS) are very SLOW and also CPU intensive.
    – svin83
    Commented May 28, 2022 at 0:46
  • The read and write raw options suspicious for me. What happened when you switch it off? (also the read size to the default (16384)...)
    – K-attila-
    Commented May 30, 2022 at 9:57

2 Answers 2

0

https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html

aio read size (S)

If this integer parameter is set to a non-zero value, Samba will read from files
asynchronously when the request size is bigger than this value.
Note that it happens only for non-chained and non-chaining reads and when
not using write cache.

The only reasonable values for this parameter are 0 (no async I/O) and 1 (always do async I/O).

same is said for aio write side

I'd recommend removing those two from your smb.conf. I've never used these two in my smb.conf.

I get 112 MB/sec read/write speed all day long over a 1gbps wired network to/from win10 pc, for reference here is my smb.conf global section

[global]
    workgroup = SAMBA
    security = user

    passdb backend = tdbsam

    printing = bsd
    printcap name = /dev/null
    load printers = no
    disable spoolss = yes
  # cups options = raw

    smb encrypt = required
    client min protocol = SMB3
    client max protocol = SMB3

in RHEL 7, the tuned profiles for reference

> tuned-adm list
    Available profiles:
    - atomic-guest                - Optimize virtual guests based on the Atomic variant
    - atomic-host                 - Optimize bare metal systems running the Atomic variant
    - balanced                    - General non-specialized tuned profile
    - cpu-partitioning            - Optimize for CPU partitioning
    - default                     - Legacy default tuned profile
    - desktop                     - Optimize for the desktop use-case
    - desktop-powersave           - Optmize for the desktop use-case with power saving
    - enterprise-storage          - Legacy profile for RHEL6, for RHEL7, please use throughput-performance profile
    - hpc-compute                 - Optimize for HPC compute workloads
    - laptop-ac-powersave         - Optimize for laptop with power savings
    - laptop-battery-powersave    - Optimize laptop profile with more aggressive power saving
    - latency-performance         - Optimize for deterministic performance at the cost of increased power consumption
    - mssql                       - Optimize for MS SQL Server
    - network-latency             - Optimize for deterministic performance at the cost of increased power consumption, focused on low latency network performance
    - network-throughput          - Optimize for streaming network throughput, generally only necessary on older CPUs or 40G+ networks
    - oracle                      - Optimize for Oracle RDBMS
    - powersave                   - Optimize for low power consumption
    - server-powersave            - Optimize for server power savings
    - spindown-disk               - Optimize for power saving by spinning-down rotational disks
    - throughput-performance      - Broadly applicable tuning that provides excellent performance across a variety of common server workloads
    - virtual-guest               - Optimize for running inside a virtual guest
    - virtual-host                - Optimize for running KVM guests
Current active profile: latency-performance
14
  • Thanks for the suggestion, I tried without the aio settings, and also replacing the whole global section with yours, I restarted the system, not just the service, but I get the same result. With a different share it is as fast as it can be (up to 115 MB/s), the problem occurs with the share "/media/hdd" which is the md0 RAID array.
    – S-Zoli
    Commented Sep 14, 2021 at 16:20
  • re-reading your 4 row chart above, FTP over LAN to the md0 results in 115 MB/sec but using samba over the same LAN results in 45 MB/sec to the md0 ? And you've already tested samba from a different disk (sda) ? I don't know how or why a sftwr md0 raid would affect network speed over samba.
    – ron
    Commented Sep 14, 2021 at 18:03
  • That's why I'm puzzled. RAID works fine, Samba works fine, but together I get slow transfer speeds and very high iowait.
    – S-Zoli
    Commented Sep 14, 2021 at 18:05
  • i use RHEL 7.9, and they have tuned. What comes to mind (in RHEL) is doing tuned-adm profile network-latency or some other relevant power profile to see if that helps; I don't know much about UBUNTU.
    – ron
    Commented Sep 14, 2021 at 18:06
  • one other recommendation- make a single ~2gb tar file and use that as the one transfer file to measure your transfer speeds if you aren't already, as opposed to copying a ~2gb folder with many small files. I've seen transfer speeds (not just on samba) plummet when the transfer is a folder full of thousands of files each of which are less than 1 MB.
    – ron
    Commented Sep 14, 2021 at 18:15
0

FTP is extremely fast and efficient compared to SMB when transferring large files. It can be difficult when it comes to small files, but overall, the speed of the FTP file transferring protocol is better. The use of short messages in SMB makes it sensible to network latency, which can decrease the speed.

From: https://cloudinfrastructureservices.co.uk/ftp-vs-smb-whats-the-difference-performance-speed-security/#:~:text=FTP%20is%20extremely%20fast%20and,which%20can%20decrease%20the%20speed.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .