we have observed that the scheduling on the DENVER cores seems to be very different from the scheduling on the ARM cores. Specifically it seems that threads are not migrated between the two cores even when the thread affinity is set to 0x6 (both DENVER cores) and one core is overloaded while the other is idling. We know that it is possible to pin threads to specific cores, but this doesn’t scale to all situations.
As an example, we have 3 tasks running doing the same job for 3 cameras running in parallel. The three tasks would easily fit to the 2 cores, if threads would be migrated between those. However, with the hard pinning approach, this is made much more difficult because for that we’d need to fit 2 tasks on one single core.
Therefore we have the following questions:
- What are the reasons for inhibiting automatic thread migration?
- We’re wondering whether there is a possibility to configure the scheduling behaviour on the DENVER cores for being more in line with “normal” linux scheduling?
- What else would you recommend for solving above situation (apart from “use the ARM cores”)?
Do you see the word “isolcpus=1,2” in: cat /proc/cmdline
What this does is disable the cores unless something is manually forced on those cores. Basically this is reserving the cores for special use, and the scheduler would not put anything there unless manually migrated through affinity.
In the kernel command line most options use the last occurring instance of that option if the option is listed multiple times (there are some exceptions to that rule, e.g., consoles can have both regular and serial console specifications). This means if you were to add this at the end of the kernel parameters, then that option would be neutralized (this won’t matter if the option already was not present): isolcpus=
(the argument with no options)
In “/boot/extlinux/extlinux.conf” note that the boot entry has the key/value pair “APPEND”, which appends to the kernel command line. It is a space-delimited value on the right hand side. Thus, this would neutralize the isolation of Denver core CPUs: APPEND ...whatever... isolcpus=
(just add a space and “isolcpus=” to the end of the line…beware that this can be a single very long line and does not have any newline in it)