How to Write Bits to Registers in Order to Disable Cache?

Autonomous Machines Jetson & Embedded Systems Jetson TK1
#1

I have been looking for a way to disable L1 and L2 caching for awhile now for benchmarking purposes. There seems to be a way to do this on the ARM Cortex-A15 by setting the 2nd bit (C bit) of the the System Control Register to 0.

Here is the some ARM documentation for quick reference:
Cache Disabled Behavior: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0438f/BABHEJFF.html
System Control Register: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0438f/BABJAHDA.html

From what I could find, there seems to be two ways to accomplish this (please correct me if I’m wrong):

  1. Write some “driver-like” code that accesses some device file (usually located in /dev) to get access to the physical addresses to flip the System Control Register C bit.

  2. Create a kernel module and write some assembly code that will set the System Control Register C bit.

I would much rather do option 1 (if at all possible) as it seems a little less work than option 2. When viewing the /proc/iomem file, I get a list of physical addresses existing on hardware:

12100000-320fffff : PCIe PREFETCH MEM Space
  12100000-121fffff : PCI Bus 0000:01
    12100000-12103fff : 0000:01:00.0
      12100000-12103fff : r8169
32100000-3fffffff : PCIe MEM Space
  32100000-321fffff : PCI Bus 0000:01
    32100000-32100fff : 0000:01:00.0
      32100000-32100fff : r8169
50000000-50033fff : /host1x
50060000-50060fff : tegra-hier-ictlr
  50060000-50060fff : tegra-hier-ictlr
54080000-540bffff : /host1x/vi
54200000-5423ffff : regs
  54200000-5423ffff : tegradc
54240000-5427ffff : regs
  54240000-5427ffff : tegradc
54280000-542bffff : hdmi_regs
  54280000-542bffff : tegradc
54300000-5433ffff : dsi_regs
  54300000-5433ffff : ganged_dsia_regs
    54300000-5433ffff : tegradc
54340000-5437ffff : /host1x/vic
54400000-5443ffff : ganged_dsib_regs
544c0000-544fffff : /host1x/msenc
54500000-5453ffff : /host1x/tsec
54600000-5463ffff : /host1x/isp@54600000
54680000-546bffff : /host1x/isp@54680000
57000000-57ffffff : /host1x/gk20a
58000000-58ffffff : /host1x/gk20a
60004800-600048ff : tegra-hier-ictlr
  60004800-600048ff : tegra-hier-ictlr
60005070-60005077 : tegra_wdt.0
  60005070-60005077 : tegra_wdt
60005100-6000511f : tegra_wdt.0
  60005100-6000511f : tegra_wdt
6000c000-6000c2ff : ahbarb
6000d000-6000dfff : /gpio@6000d000
60020000-600213ff : /dma@60020000
70000820-70000827 : 70000868.pinmux
70000868-700009cb : 70000868.pinmux
70003000-70003433 : 70000868.pinmux
70006000-7000603f : /serial@70006000
70006040-7000607f : /serial@70006040
70006200-7000623f : /serial@70006200
70006300-7000631f : serial
7000a000-7000a0ff : /pwm@7000a000
7000c000-7000c0ff : /i2c@7000c000
7000c300-7000c3ff : dtv
  7000c300-7000c3ff : dtv
7000c400-7000c4ff : /i2c@7000c400
7000c500-7000c5ff : /i2c@7000c500
7000c700-7000c7ff : /i2c@7000c700
7000d000-7000d0ff : /i2c@7000d000
7000d100-7000d1ff : /i2c@7000d100
7000d400-7000d5ff : /spi@7000d400
7000da00-7000dbff : /spi@7000da00
7000e000-7000e0ff : tegra_rtc
7000e400-7000e4ff : tegra_wdt.0
  7000e400-7000e4ff : tegra_wdt
7000f800-7000fbff : /efuse@7000f800
70019000-700197ff : mc
7001b000-7001b7ff : tegra-emc
70030000-7003ffff : tegra30-hda
  70030000-7003ffff : tegra30-hda
700b0400-700b05ff : sdhci-tegra.2
  700b0400-700b05ff : mmc1
700b0600-700b07ff : sdhci-tegra.3
  700b0600-700b07ff : mmc0
700e3000-700e30ff : mipi_cal
  700e3000-700e30ff : tegradc
70110000-701103ff : tegra_cl_dvfs
70300000-7030007f : tegra30-ahub
  70300000-7030007f : tegra30-ahub
70300800-70300fff : tegra30-ahub
  70300800-70300fff : tegra30-ahub
70301000-703010ff : tegra30-i2s.0
  70301000-703010ff : tegra30-i2s
70301100-703011ff : tegra30-i2s.1
  70301100-703011ff : tegra30-i2s
70301300-703013ff : tegra30-i2s.3
  70301300-703013ff : tegra30-i2s
70301400-703014ff : tegra30-i2s.4
  70301400-703014ff : tegra30-i2s
70302000-703021ff : tegra30-dam.0
  70302000-703021ff : tegra30-dam
70302200-703023ff : tegra30-dam.1
  70302200-703023ff : tegra30-dam
70302400-703025ff : tegra30-dam.2
  70302400-703025ff : tegra30-dam
70306000-703060ff : tegra30-spdif
  70306000-703060ff : tegra30-spdif
7d000000-7d003fff : tegra-ehci.0
  7d000000-7d003fff : tegra-udc.0
    7d000000-7d003fff : tegra-otg
      7d000000-7d003fff : tegra-udc
7d004000-7d007fff : tegra-ehci.1
7d008000-7d00bfff : tegra-ehci.2
80000000-f82fffff : System RAM
  80008000-80b5b6cb : Kernel code
  80bda000-80dd5f8b : Kernel data
f8300000-f83fffff : persistent_ram
f8400000-f84fffff : persistent_ram
f8500000-f96fffff : fbmem
f9700000-fdefffff : fbmem

These registers seem to line up with what I’m seeing on the TegraK1 TRM; however, I do not see where the System Control Register resides in /proc/iomem or the TRM. In the TRM, it lists the “System Registers” starting at 6000:C000 which coincides with “ahbarb” in /proc/iomem, but I don’t even know if what ARM calls the “System Control Register” is located within what Nvidia calls “System Registers”. If it is, then I still don’t know the address offset of the System Control Register in order to flip the C bit.

As for option 2, disabling the cache is part of the powering down sequence for the TK1. As a result, I was able to find some assembly in the kernel source code to reference if I must pursue option 2.

/*
 * Clear the SCTLR.C bit to prevent further data cache
 * allocation. Clearing SCTLR.C would make all the data accesses
 * strongly ordered and would not hit the cache.
 */
mrc	p15, 0, r0, c1, c0, 0
bic	r0, r0, #(1 << 2)	@ Disable the C bit
mcr	p15, 0, r0, c1, c0, 0
isb

I do not know if option 1 is even possible. Am I really able to write to physical addresses using a device file? If so, how? Are there alternative ways to write to the ARM’s control registers? I saw this thread that talks about reading from (but not writing to) physical addresses by remapping parts of the /dev/mem device file to userspace memory:

http://stackoverflow.com/questions/12040303/accessing-physical-address-from-user-space

I don’t know if I’m actually onto something here, or if I’m just chasing my tail. Surely someone must know how to set bits to ARM registers, even if it is buried underneath an OS.

#2

Hi aowens,

Disabling cache would break atomic operations that are implemented using ldrex/strex instructions.
What benchmarks you’re trying to run? Can you allocate uncached memory and use it for benchmarks?

Thanks

#3

Hi kayccc,

The application is for aerospace; disabling cache may increase system up-time but I would like to have some numbers/statistics on how detrimental it would be to performance, hence the reason for the benchmarks. As a result, allocating uncached memory for the benchmarks would not be useful for me since I will be needing to disable the cache anyways.

Would it be possible to disable just L1 or just L2 cache instead of both without the repercussions of breaking atomic operations? If not, would the kernel source code have any flags I could potentialy set to disable cache so that when I recompile the kernel source code, any operations that would otherwise be broken would know at boot not to use cache since it was compiled with cache disabled?

Thank you

#4

Hi aowens,

Both L1 and L2 are inner caches. So, It is not possible disable one of them.

Thanks

#5

I thought that disabling the cache was part of the boot-down procedure?

“When you clear the C bit in the CP15 System Control Register for a given processor, see System Control Register, data caching is disabled and no new cache lines are allocated to the L1 data cache and L2 cache because of requests from that processor. This is important when cleaning and invalidating the caches for power down.” (http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0438f/BABHEJFF.html)

So there must be a way to disable/bypass the cache, right?

#6

Hi aowens,

It might be possible to use the generic atomic operations and avoid issues with ldrex/strex instructions. All the instances ldrex/strex in other places need to be fixed as well, but that will be a bigger task to do.

Thanks

#7