CPU identification faulty in Multi-Core Designs (incl. solution + further implications) - BCC2

Hello,

we figured out, that the bcc_get_cpuid() function gives wrong values, once several CPUs are started.

Take the following code, which starts one core after each other, asks for the started core’s ID and does the same again, once all cores are started (compile with: sparc-gaisler-elf-gcc -qbsp=leon3 -Og -g -msoft-float -mcpu=leon3 -Wall -Wextra -pedantic -ldrv main.c):

#include <bcc/bcc.h>
#include <stdlib.h>
#include <stdio.h>

int main(void)
{
	int cpuID = bcc_get_cpuid();
    printf("I'm core %d. \n", cpuID);
    if (!cpuID) {
        for(int i = 1; i < bcc_get_cpu_count(); i++) {
            for (int j = 0; j < 100000; j++) asm("nop");
			bcc_start_processor(i);
        }
    }

    if (cpuID == bcc_get_cpu_count()-1) {
        for (int j = 0; j < 10000000; j++) asm("nop");
        for(int i = 0; i < bcc_get_cpu_count()-1; i++) {
			bcc_start_processor(i);
            for (int j = 0; j < 10000000; j++) asm("nop");
        }
    } else {
        bcc_power_down();
    }

    printf("I'm core %d. But now I think I'm core %d. \n", cpuID, bcc_get_cpuid());
    if (cpuID != bcc_get_cpu_count()-1) bcc_power_down();

 	return 0;
}

The output in a three core system looks like this:

I’m core 0.
I’m core 1.
I’m core 2.
I’m core 0. But now I think I’m core 2.
I’m core 1. But now I think I’m core 2.
I’m core 2. But now I think I’m core 2.

Changing a single line: for(int i = 1; i < bcc_get_cpu_count(); i++) { to for(int i = bcc_get_cpu_count() - 1; i > 0 ; i--) { which modifys the order in which the CPUs get started, shows us how the the function works:

I’m core 0.
I’m core 2.
I’m core 1.
I’m core 0. But now I think I’m core 1.
I’m core 1. But now I think I’m core 1.
I’m core 2. But now I think I’m core 1.

We see, the function always returns the index of the last started core.

Therefore, we propose to replace the function bcc_get_cpuid() by the following version:

static inline unsigned int bcc_get_cpuid() {
	unsigned int ret;
	asm volatile("mov %%asr17, %0\n"
	             "srl %0, 28, %0\n"
	             "and %0, 0xf, %0\n"
	           : "=r"(ret)
	);
	return ret;
}

This function works independent of which core was started last, as it reads the %ASR17 which stores the CPU’s ID in its highest bits (VHDL generic and read-only).

The fact, that bcc_get_cpuid() returns wrong values results in further interesting behavior as the function is used several times internaly by BCC2.
For example masking and unmasking interrupts gives wrong values in the interrupt controller, as the functions bcc_int_mask() and bcc_int_unmask() use the macro __bcc_cpuid which seems to be resolved as bcc_get_cpuid().
Therefore, we also had to modify their implementations.

I hope to help others with this post here and want to ask the developers to solve the bug in the next BCC release.

-Flo

Hi Flo!

I don’t think this is to be considered a bug since BCC2 just like BCC1 does not support SMP operation (where multiple CPUs run the same binary in parallel). It only supports AMP operation, where you compile a separate binary to run on each CPU and take some additional care so that they do not use the same hardware.

In order to support SMP you would have to have proper locking etc implemented in the whole runtime. Even with this patch you will run into other problems down the line. You’re probably better off using RTEMS5 since it supports true SMP operation.