11

My objective is to set up a mini server (not HTPC) with low power consumption in idle mode, yet offering nice performance when used. The focus is more on data safety than availability. In other words: quality parts, but redundancy only for storage.

Not considering myself to be biased, after some research I felt that some AMD desktop APUs would offer good value.

Questions remaining are:

  • Will the idle state of the GPU lower power consumption and unleash resources for the CPU?
  • Will Cool'n'Quiet and Turbo Core lead to the intended low power consumption in idle mode but performance under load?
  • Will Linux support this scenario as intended? Quite a few questions and forum discussions seem to suggest that this is not necessarily the case.
13
+50

[Edit: Concluding thoughts regarding the processor choice]

  • AMD vs AMD:
    • Richland does a much better job than Trinity here.
    • Kaveri cannot compete with Richland's idle mode power dissipation (at least for now).
    • The GPU of the A10-6700 may be overrated, but it's a bit sad it won't be used much. Some algorithms may be able to deploy its computational power. No idea how that will affect the processor's power consumption, though.
    • I suspect the A10-6790K to be the same processor as the A10-6700 with just a different parameter set for Turbo Core boosts. If this is true, the A10-6790K will be able to boost longer and/or provide higher frequencies in the long term due to its higher TDP. But you'll need a different CPU fan for that (think space and temperature/life span).
  • AMD A10-6700 vs Intel Core i3-3220:
    • The A10-6700 has a lot more GPU power, which is unused here.
    • The i3-3220 has a lower idle mode power dissipation.
    • While in typical benchmarks the i3-3220 is faster for computations, I cannot see how its two hyper-threading cores would be able to handle parallel requests (say, to a database with web frontends) as fast as four fully featured cores (at least when assuming some serious caching). Didn't find any serious benchmarks, though -- Only some indications.

[Edit: The free radeon driver's bapm parameter is set by default for Kaveri, Kabini and desktop Trinity, Richland systems as of Linux 3.16]

See [pull] radeon drm-fixes-3.16.

However, regarding 3.16 based Debian, the defaults don't (yet?) seem to work, while the boot parameter does. See How to set up a Debian system (focus on 2D or console/server) with an AMD Turbo Core APU for maximum energy and computing efficiency?

[Edit: The free radeon driver will soon have a bapm parameter]

Since the bottom line of the below is to use a patched version of the free radeon driver with your APU to support Turbo Core and get the most out of it (except 3D graphics that is) if you can (enabling bapm can lead to instabilities in some configurations), it's great news that future versions of radeon will have a parameter to enable bapm.

[Original post follows]

AMD A10-6700 (Richland) APU Experience

Processor Choice

My first PC was a 486DX2-66 set up from dozens of 3.5" floppy disks containing Slackware source packages. Sice then, a lot of things have changed, and a lot of industries currently seem to be in the phase where they still increase the number of product variants.

This circumstance and some of AMD's unfortunate decisions in the recent past haven't made it easier for me to decide on a platform for a mini server. But finally, I decided that the A10-6700 would be a good choice:

  • Several reviews have shown that a (still widely unavailable) Kaveri will consume more power in idle mode than a Richland or a Trinity
  • The advantage of the Richland A10-6700 over the Trinity A10-5700 seem to be significant: Lower lowest and higher highest frequency, more fine-grained Turbo Core (considering also temperature -- quite an advantage when the GPU will be idle)
  • The GPU of the A10-6700 is said to be overrated (marketing-driven naming) and the APU's pricing seems fair

Other Components and Setup

Despite the countless processors to choose from, there aren't many Mini-ITX boards available. The ASRock FM2A78M-ITX+ appeared to be a reasonable choice. The test was done with firmware V1.30 (no updates available as I write this).

Only 80% of a power supply's nominal output should be consumed. On the other hand, many fail to work efficiently below 50% load. It's very difficult to find an energy efficient power supply for a system with an estimated power dissipation range of 35W to 120W. I conducted these tests with a Seasonic G360 80+ Gold because it outperforms most competitors regarding efficiency at low loads.

Two 8GB DDR3-1866 RAMs (configured as such -- which does make a difference as compared to 1333), one SSD drive and a PWM controleld quality CPU fan were also part of the test setup.

The measurements were made using an AVM Fritz!DECT 200 which has been reported to perform accurate measurements. Still, plausibility was validated using an older no-name device. No inconsistencies could be identified. The measured system power dissipation will include the power supply's reduced efficiency for lower loads.

A [W]QHD screen was connected via HDMI.

The initial shared memory for the GPU was set to 32M in the UEFI BIOS. Also, the Onboard GPU was selected as Primary, and IOMMU was enabled.

No X or other graphical system was installed or configured. Video output was restricted to console mode.

Basics

There are a few things one needs to know.

  • While the decision about Cool'n'Quiet is made by software outside of the processors, Turbo Core is a decision made autonomously by an additional microcontroller on the APU (or CPU).
  • Many tools as well as /proc and /sys places don't report Turbo Core activity. cpufreq-aperf, cpupower frequency-info and cpupower monitor do, but only after modprobe msr.

Test Case Group 1: Linux + radeon

I started with fresh Arch Linux (installer 2014.08.01, kernel 3.15.7). Key factor here is the presence of acpi_cpufreq (kernel CPU scaling) and radeon (kernel GPU driver) and the easy way to patch radeon.

Test Case 1.1: BIOS TC on - CnQ on / Linux OnDemand - Boost

UEFI BIOS Turbo Core Setting............................ Enabled
UEFI BIOS Cool'n'Quiet Setting.......................... Enabled
/sys/devices/system/cpu/cpufreq/boost................... 1
/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor... ondemand 
"cpupower frequency-info" Pstates........ 4300 4200 3900 3700 3400 2700 2300 1800
Observed "/proc/cpuinfo" cpu MHz range... 1800 - 3700
Observed "cpupower monitor" Freq range... 1800 - 3700
/sys/kernel/debug/dri/0/radeon_pm_info... power level 0
Load           | Core Freqs
---------------+-----------
stress --cpu 1 | 1 x 3700
stress --cpu 2 | 2 x 3700
stress --cpu 3 | 3 x 3700
stress --cpu 4 | 4 x 3700

Test Case 1.2: BIOS TC on - CnQ on / Linux Performance - Boost

UEFI BIOS Turbo Core Setting............................ Enabled
UEFI BIOS Cool'n'Quiet Setting.......................... Enabled
/sys/devices/system/cpu/cpufreq/boost................... 1
/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor... performance 
"cpupower frequency-info" Pstates........ 4300 4200 3900 3700 3400 2700 2300 1800
Observed "/proc/cpuinfo" cpu MHz range... 3700
Observed "cpupower monitor" Freq range... 2000 - 3700
/sys/kernel/debug/dri/0/radeon_pm_info... power level 0
Load           | Core Freqs
---------------+-----------
stress --cpu 1 | 1 x 3700
stress --cpu 2 | 2 x 3700
stress --cpu 3 | 3 x 3700
stress --cpu 4 | 4 x 3700

Test Case Group 1 Summary

Turbo Core based boosts are impossible in this scenario because the radeon driver currently disables the bapm flag due to stability issues in some scenarios. Therefore, further testing was skipped.


Test Case Group 2: Linux + bapm-patched radeon

In order to enable bapm, I started with a fresh Arch Linux (installer 2014.08.01, kernel 3.15.7), got me the core linux package via ABS (3.15.8), edited the PKGBUILD to use pkgbase=linux-tc, pulled the sources with makepkg --nobuild, changed pi->enable_bapm = true; in trinity_dpm_init() in src/linux-3.15/drivers/gpu/drm/radeon/trinity_dpm.c, and compiled it with makepkg --noextract. Then, I installed it (pacman -U linux-tc-headers-3.15.8-1-x86_64.pkg.tar.xz and pacman -U linux-tc-3.15.8-1-x86_64.pkg.tar.xz) and updated GRUB (grub-mkconfig -o /boot/grub/grub.cfg but, of course, YMMV).

As a result, I was given the choice to boot linux or linux-tc, and the following tests refer to the latter.

Test Case 2.1: BIOS TC on - CnQ on / Linux OnDemand - Boost

UEFI BIOS Turbo Core Setting............................ Enabled
UEFI BIOS Cool'n'Quiet Setting.......................... Enabled
/sys/devices/system/cpu/cpufreq/boost................... 1
/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor... ondemand 
"cpupower frequency-info" Pstates........ 4300 4200 3900 3700 3400 2700 2300 1800
Observed "/proc/cpuinfo" cpu MHz range... 1800 - 3700
Observed "cpupower monitor" Freq range... 1800 - 4300
/sys/kernel/debug/dri/0/radeon_pm_info... power level 0
Load           | Core Freqs
---------------+-----------------
stress --cpu 1 | 1 x 4300
stress --cpu 2 | 2 x 4200 .. 4100
stress --cpu 3 | 3 x 4100 .. 3900
stress --cpu 4 | 4 x 4000 .. 3800

Test Case 2.2: BIOS TC on - CnQ on / Linux Performance - Boost

UEFI BIOS Turbo Core Setting............................ Enabled
UEFI BIOS Cool'n'Quiet Setting.......................... Enabled
/sys/devices/system/cpu/cpufreq/boost................... 1
/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor... performace
"cpupower frequency-info" Pstates........ 4300 4200 3900 3700 3400 2700 2300 1800
Observed "/proc/cpuinfo" cpu MHz range... 3700
Observed "cpupower monitor" Freq range... 2000 - 4300
/sys/kernel/debug/dri/0/radeon_pm_info... power level 0
Load           | Core Freqs
---------------+-----------------
stress --cpu 1 | 1 x 4300
stress --cpu 2 | 2 x 4200 .. 4100
stress --cpu 3 | 3 x 4100 .. 3900
stress --cpu 4 | 4 x 4000 .. 3800

Test Case 2.3: BIOS TC on - CnQ on / Linux OnDemand - No Boost

UEFI BIOS Turbo Core Setting............................ Enabled
UEFI BIOS Cool'n'Quiet Setting.......................... Enabled
/sys/devices/system/cpu/cpufreq/boost................... 0
/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor... ondemand
"cpupower frequency-info" Pstates........ 4300 4200 3900 3700 3400 2700 2300 1800
Observed "/proc/cpuinfo" cpu MHz range... 1800 - 3700
Observed "cpupower monitor" Freq range... 1800 - 3700
/sys/kernel/debug/dri/0/radeon_pm_info... power level 1
Load           | Core Freqs
---------------+-----------
stress --cpu 1 | 1 x 3700
stress --cpu 2 | 2 x 3700
stress --cpu 3 | 3 x 3700
stress --cpu 4 | 4 x 3700

Test Case 2.4: BIOS TC on - CnQ on / Linux Performance - No Boost

UEFI BIOS Turbo Core Setting............................ Enabled
UEFI BIOS Cool'n'Quiet Setting.......................... Enabled
/sys/devices/system/cpu/cpufreq/boost................... 0
/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor... performace
"cpupower frequency-info" Pstates........ 4300 4200 3900 3700 3400 2700 2300 1800
Observed "/proc/cpuinfo" cpu MHz range... 3700
Observed "cpupower monitor" Freq range... 2000 - 3700
/sys/kernel/debug/dri/0/radeon_pm_info... power level 1
Load           | Core Freqs
---------------+-----------
stress --cpu 1 | 1 x 3700
stress --cpu 2 | 2 x 3700
stress --cpu 3 | 3 x 3700
stress --cpu 4 | 4 x 3700

Test Case 2.5: BIOS TC off - CnQ on / Linux OnDemand - Boost

UEFI BIOS Turbo Core Setting............................ Disabled
UEFI BIOS Cool'n'Quiet Setting.......................... Enabled
/sys/devices/system/cpu/cpufreq/boost................... 1
/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor... ondemand 
"cpupower frequency-info" Pstates........ 4300 4200 3900 3700 3400 2700 2300 1800
Observed "/proc/cpuinfo" cpu MHz range... 1800 - 3700
Observed "cpupower monitor" Freq range... 1800 - 3700
/sys/kernel/debug/dri/0/radeon_pm_info... power level 0

In other words, if Turbo Core is disabled in the BIOS, the patched radeon will not turn it on.

Test Case 2.6: BIOS TC on - CnQ off / Linux n/a

UEFI BIOS Turbo Core Setting............................ Enabled
UEFI BIOS Cool'n'Quiet Setting.......................... Disabled
/sys/devices/system/cpu/cpufreq/boost................... n/a
/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor... n/a
"cpupower frequency-info" Pstates........ 4300 4200 3900 3700 3400 2700 2300 1800
Observed "/proc/cpuinfo" cpu MHz range... 3700
Observed "cpupower monitor" Freq range... 2000 - 4300
/sys/kernel/debug/dri/0/radeon_pm_info... power level 0
Load           | Core Freqs
---------------+-----------------
stress --cpu 1 | 1 x 4300
stress --cpu 2 | 2 x 4100 .. 4000
stress --cpu 3 | 3 x 4000 .. 3800
stress --cpu 4 | 4 x 3900 .. 3700

With Cool'n'Qiet disabled, the Linux kernel will not offer any governor choice and (wrongly) assume that the cores run at a fixed frequency. Interestingly, the resulting Turbo Core frequencies are the worst of all tested combinations in Test Case Group 2.

Test Case Group 2 Summary

With the patched radeon driver, Turbo Core works. No instabilities (which are the reason why bapm aka Turbo Core is disabled there) have been seen so far.


Test Case Group 3: Linux + fglrx (catalyst)

I started with a fresh Ubuntu (14.04 Server, kernel 3.13) installation, which I see as comparable to the Arch Linux (installer 2014.08.01, kernel 3.15.7) due to the presence of acpi_cpufreq (kernel CPU scaling) and radeon (kernel GPU driver). The reason for switching to Ubuntu is the easy installation of fglrx. I validated the power consumption and behaviour with the fresh installation, which uses radeon.

I installed fglrx from the command line (sudo apt-get install linux-headers-generic, sudo apt-get install fglrx) and rebooted the system. The change from radeon to fglrx is immediately obvious both regarding console appearance (fglrx: 128 x 48, radeon: much higher) and idle mode power consumption (fglrx: 40W, radeon: 30W). But Turbo Core works right away.

Test Case 3.1: BIOS TC on - CnQ on / Linux OnDemand - Boost

UEFI BIOS Turbo Core Setting............................ Enabled
UEFI BIOS Cool'n'Quiet Setting.......................... Enabled
/sys/devices/system/cpu/cpufreq/boost................... 1
/sys/devices/system/cpu/cpu*/cpufreq/scaling_governor... ondemand 
"cpupower frequency-info" Pstates........ 4300 4200 3900 3700 3400 2700 2300 1800
Observed "/proc/cpuinfo" cpu MHz range... 1800 - 3700
Observed "cpupower monitor" Freq range... 1800 - 4300
/sys/kernel/debug/dri/0/radeon_pm_info... n/a
Load           | Core Freqs
---------------+----------------------------
stress --cpu 1 | 1 x 4300
stress --cpu 2 | 2 x 4200 .. 3900 (core chg)
stress --cpu 3 | 3 x 4100 .. 3700
stress --cpu 4 | 4 x 4000 .. 3600

The fglrx behaviour is definitely interesting. When 'stress --cpu 2' was called for any of the tes cases in any test case group, the two loaded cores were always located on separate modules. But with fglrx, a sudden reallocation occured such that a single module was used (which saves quite some power see below). After some time, the loaded core moved back to the other module. This was not seen with radeon. Could it be that fglrx manipulates core affinity of processes?

Test Case Group 3 Summary

The advantage of fglrx is that it enables Turbo Core right away, without any need to patch it.

Because fglrx wastes 10 to 12W for the GPU in our scenario on a chip with 65W TDP, the overall results regarding available core speeds are unimpressive. Therefore, no further tests were conducted.

Also from an engineering point of view, the behaviour of fglrxappears to be a bit sad. Reallocating one of two busy cores to the other module to maintain a higher frequency may or may not be a good idea, because before that step, both cores had a L2 cache of their own, while afterwards, they have to share one. Whether fglrx considers any metrics (such as cache hit misses) to support its decision will have to be clarified separately, but there are other reports about its abrupt behaviour.


Summary of Power Consumption

Some of the delta values in the following table get slightly worse as the temperature rises; one might say the PWM controlled fan and the chip both play a role there.

System @State / ->Transition Delta | System Power Dissipation
-------------------------------------+-------------------------
  @BIOS                            |  @ 95 .. 86W
  @Bootloader                      |  @108 .. 89W
  @Ubuntu Installer Idle           |  @ 40W
  @Linux radeon Idle ondemand      |  @ 30W
  @Linux radeon Idle performance   |  @ 30W
  @Linux fglrx Idle ondemand       |  @ 40W
  1 Module 1800 -> 3700            |  + 13W
  1 Module 1800 -> 4300            |  + 25W
  1 Core 1800 -> 3700              |  +  5W
  1 Core 1800 -> 4300              |  + 10W
  "radeon" Video Out -> Disable    |  -  2W
  'fglrx" Video Out -> Darken      |  +- 0W
  @Linux radeon Maximum            |  @103 .. 89W
  @Linux fglrx Maximum             |  @105 .. 92W
  • There seems to be more to Turbo Core (at least with Richland APUs) than expected: There is no noticeable difference in power dissipation when the "ondemand" scaling governor is in place as compared to when the "performance" governor is in place. Althouth /proc/cpuinfo will always report 37000MHz under the performance governor, cpupower monitor will reveal that the cores actually do slow down. In some cases, frequencies as low as 2000MHz were shown; it's possible that 1800MHz will internally be used as well.
  • The A10-6700 consists of two modules with two cores each. If e.g. two cores are idle and two cores are busy and get accelerated, the system behaviour will be different depending on whether the busy cores are located on the same module or not.
    • Accelerating a module is more energy-intensive than accelerating a core.
    • The L2 cache is assigned per module.
  • The difference between the power dissipation of two cores accelerating on the same module vs on different modules was determined by replacing stress --cpu 2 (which always led to a distribution amongst the two modules) by taskset -c 0 stress --cpu 1andtaskset -c 1 stress --cpu 1.
  • The A10-6700 seems to have a total power dissipation limit for the APU (92W together with the other components) with tiny bit reserved for the GPU alone (3 W). With radeon, it will allow for more for a short period and reduce to the maximum very smoothly, while with fglrx, it has been observed that these limits are exceeded more significantly and power dissipation is then reduced abruptly.
  • While many people claim that the delay in Kaveri availability is intended by AMD because it would kill their current APUs, I beg to differ. The Richland A10 has demonstrated an excellent power management, and the Kaveri cannot compete with its low idle state power consumption (Kaveri's chip complexity is almost twice that of Richland's, so it will take another one or two development steps).

Overall Conclusion

  • Including temperature in the Turbo Core logic (as is reported for the Trinity -> Richland step) seems to make sense and appears to work well, as can be seen by the reduction in pwoer dissipation in BIOS and Bootloader over time.
  • For the cosole/server scenario, the A10-6700 supports 4 cores @ 3700MHz (3800MHz with Turbo Core) over the long term, at least with the radeon driver. There's probably not much chance to maintain this performance level when the GPU gets some work to do.
  • It would seem that the 65W TDP can be permanently exceeded slightly under full load, but it's hard to tell as the power supply has a lower efficiency at 30W. Since there are clear indications that the temperature is considered (a peak power dissipation of almost 110W was observed before it started to be reduced to 90W, and also 2 cores at 4300 MHz were reported for some time), investing in APU cooling may be a good idea. However, mainboards limited to 65W TDP will only be able to supply so much current, so there will definitely be a hard limit imposed by the APU.
  • If you intend to use a Richland APU for computing under Linux, you definitely want to use a patched radeon driver (if you do not encounter instabilities -- specifically in conjunction with the enabling of Dynamic Power Management). Otherwise, you'll not get full value.
  • Oddly enough, it seems that the best setup would be to enable both Turbo Core and Cool'n'Quiet in the BIOS but then choose the performance scaling governor -- at least if your APU behaves like the one tested here. You'll have the same power consumption as with ondemand but faster frequency scaling and less kernel overhead to make the scaling decision.

Acknowledgements

Special thanks goes to Alex Deucher, who significantly pushed me into the right direction over at bugzilla.kernel.org.

I am impressed by the quality of the free radeon driver and would like to thank the whole team for maintaining this piece of software, which appears to be thoughtfully engineered. If radeon would not behave as it does, my decision in favour of the A10-6700 would have been substantially wrong.

  • As an Arch user who is interested in idle power consumption efficiency I found this article to be one of the best resources I've seen for optimizing AMD APU's on Arch. Thanks! This should be posted in the Arch wiki. – b10hazard Feb 24 '15 at 13:10
  • Thank you for your feedback, @b10hazard, and this sounds like a good idea. What would be the steps to integrate this into the Arch Wiki? I am new to Arch; I was more on the Debian side until recently. – Run CMD Feb 24 '15 at 18:45
  • I'm not sure. Not many people are interested in low power consumption on their PC's and even fewer have acquired the wealth of information you have on the subject. It would be a shame not to incorporate some of this into the wiki. Perhaps you could ask someone on the forums? Wish I could be of more help, I've never created a page on the wiki, I've only edited existing pages. – b10hazard Mar 4 '15 at 21:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.