Considerations about multicore programming

JoaoLopesF
Posts: 59
Joined: Thu Aug 17, 2017 5:40 pm

Considerations about multicore programming

Postby JoaoLopesF » Mon Aug 20, 2018 4:09 pm

Hi,

When I was starting with ESP-IDF, I simply created a task on CPU1 (with a delay of 1 second)
and thought I was using the 2 CPUs of ESP32.

But I was wrong and almost all processing was still on CPU 0 ..

That is why I created this post.

Considerations about multicore programming with ESP-IDF and FreeRTOS:

- All ESP-IDF protocol stuff (UART, WiFi, BLE, etc.) run in CPU 0 (PRO_CPU_NUM)

- Generally what you want is to leave CPU 0 for this and that our application runs entire on CPU 1 (APP_CPU_NUM)

- The project should consider if CONFIG_FREERTOS_UNICORE is set, not to use CPU 1 (which may not even exist)

- Only the code in the FreeRTOS task that was created to run on CPU 1,
  is what actually runs on CPU1, with the exception of hardware timers that can run on CPU 1 as well.
If this task / timer has a delay of 1 second, for example, the use of CPU 1 can be minimal

- All ESP-IDF protocol stuff (UART, WiFi, BLE, etc) have callbacks,
for example when receiving data from BLE, to call the code in your project.
And this continues on CPU 0. For the project processing to be on CPU 1,
this should implement FreeRTOS task and queue / semaphore
(it is a queue too) or task notification to run these callbacks in CPU 1

- ISR codes, such as GPIO interrupts, if they have some heavier processing,
should use FreeRTOS queue / semaphore or Task notification
for a task assigned to it. If possible use CPU 1 for this.
For GPIO interrupts, I use task notifications.

- esp_timer has a task for calbacks, and only runs on CPU 0,
  so this can not be used for multicore optimizations

- Exist a sdkconfig CONFIG_ESP32_WIFI_TASK_PINNED_TO_CORE_1 to WiFi use task in CPU 1,
but I dont use it yet

To get at these considerations, I've made macros for ESP-IDF logging,
putting the Core Id in each log: viewtopic.php?f=18&t=6567.

An example: I recently made an example project to be used in BLE communication with mobile devices.
Link: viewtopic.php?f=18&t=6806

This project uses these macros and implements everything I've said.

The BLE callback task can be disabled and generates the following logs:

Code: Select all

I (0) cpu_start: App cpu up.
I (293)P(06) heap_init: Initializing. RAM available for dynamic allocation:
I (296)P(03) heap_init: At 3FFAFF10 len 000000F0 (0 KiB): DRAM
I (299)P(03) heap_init: At 3FFCA800 len 00015800 (86 KiB): DRAM
I (302)P(03) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (305)P(03) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (308)P(03) heap_init: At 40092770 len 0000D890 (54 KiB): IRAM
I (311)P(03) cpu_start: Pro cpu start user code
I (323)P(12) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (325)P(02) main: (app_main)(C0) Initializing
I (325)P(00) util: (esp32Initialize)(C0) Initalizing Esp32 ...
I (345)P(20) util: (esp32Initialize)(C0) Esp32 initialized
D (345)P(00) peripherals: (peripheralsInitialize)(C0) Initializing peripherals ...
D (345)P(00) peripherals: (gpioInitialize)(C0) Initializing GPIO ...
I (355)P(10) gpio: GPIO[4]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 0| Pulldown: 1| Intr:1 
I (355)P(00) gpio: GPIO[16]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 0| Pulldown: 1| Intr:3 
I (365)P(10) gpio: GPIO[17]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 0| Pulldown: 1| Intr:3 
I (365)P(00) gpio: GPIO[5]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
I (365)P(00) gpio: GPIO[12]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 1| Intr:0 
D (375)P(10) peripherals: (gpioInitialize)(C0) GPIO initalized
D (375)P(00) peripherals: (adcInitialize)(C0) Initializing ADC ...
D (385)P(10) peripherals: (adcInitialize)(C0) ADC Initialized
D (385)P(00) peripherals: (peripheralsInitialize)(C0) Peripherals initialized
I (385)P(00) ble_server: (initialize)(C0) Initializing BLE Server - device name=Esp32_Device_
I (395)P(10) BTDM_INIT: BT controller compile version [a9156c1]

I (395)P(00) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (455)P(60) phy: phy_version: 3960, 5211945, Jul 18 2018, 10:40:07, 0, 0
I (725)P!(270) ble_uart_server: (ble_uart_server_Initialize)(C0) BLE UART server initialized, device name Esp32_Device_7CCE
I (725)P(00) ble_server: (initialize)(C0) BLE initialized
I (725)P(00) ble-server: (bleInitialize)(C0) BLE Server initialized!
I (725)P(00) main: (main_Task)(C1) Starting main Task
D (735)P(10) main: (appInitialize)(C1) Initializing ...
D (735)P(00) main: (appInitialize)(C1) Initialized
I (3605)P!(2870) ble_server: (processEventConnection)(C0) BLE client connected
D (3615)P(10) ble-server: (onConnect)(C0) Ble connected!
I (3695)P(80) ble_server: (bleCallbackMTU)(C0) BLE MTU changed to 185
V (4505)P!(810) ble_server: (bleCallbackReceiveData)(C0) BLE received [4] : 01:\n
D (4515)P(10) ble_server: (bleCallbackReceiveData)(C0) BLE line message received: 01:
V (4515)P(00) main: (processBleMessage)(C0) Code -> 1 Message -> 01:
V (4515)P(00) main: (debugInitial)(C0) Debugging is on now
V (4515)P(00) main: (debugInitial)(C0) Firmware device version: 0.1.0
D (4525)P(10) main: (appInitialize)(C0) Initializing ...
D (4525)P(00) main: (appInitialize)(C0) Initialized
D (4525)P(00) main: (notifyMainTask)(C0) action=1
D (4525)P(00) main: (main_Task)(C1) Notification received -> 1
V (4525)P(00) ble_server: (send)(C0) BLE message [13] -> 01:0.1.0:Y:Y\n
D (4525)P(00) main: (main_Task)(C1) Reseted time
D (4535)P(10) main: (checkEnergyVoltage)(C0) vbat=3397 diff=3397
V (4535)P(00) ble_server: (send)(C0) BLE message [15] -> 10:EXT:Y:3397:\n
V (6455)P!(1920) ble_server: (bleCallbackReceiveData)(C0) BLE received [7] : 11:ALL\n
D (6455)P(00) ble_server: (bleCallbackReceiveData)(C0) BLE line message received: 11:ALL
V (6465)P(10) main: (processBleMessage)(C0) Code -> 11 Message -> 11:ALL
V (6465)P(00) main: (sendInfo)(C0) type=ALL
V (6465)P(00) main: (sendInfo)(C0) rom_phy_get_vdd33=6743
D (6465)P(00) main: (checkEnergyVoltage)(C0) vbat=3393 diff=4
V (6475)P(10) ble_server: (send)(C0) BLE message [15] -> 10:EXT:Y:3393:\n
V (6475)P(00) ble_server: (send)(C0) BLE message [237] -> 11:ESP32:*** Chip Info#* Model; 1#* Revision; 1#* Cores; 2#* FreeRTOS unicore ?; No#* ESP-IDF;#  v3.1-beta1-216-gd91c4251-dirty#*** BLE info#* Device name; Esp32_Device_7CCE#* Mac-address; 30;AE;A4;07;7C;CE#\n11:FMEM:189416\n11:VDD33:6743\n
V (6485)P(10) ble_server: (send)(C0) BLE sending part [180] -> 11:ESP32:*** Chip Info#* Model; 1#* Revision; 1#* Cores; 2#* FreeRTOS unicore ?; No#* ESP-IDF;#  v3.1-beta1-216-gd91c4251-dirty#*** BLE info#* Device name; Esp32_Device_7CCE#* Mac- [max=180]
V (6495)P(10) ble_server: (send)(C0) BLE sending part [57] -> address; 30;AE;A4;07;7C;CE#\n11:FMEM:189416\n11:VDD33:6743\n [max=180]
D (9535)P!(3040) main: (main_Task)(C1) * Secs=5 | sensors: vext=Y charging=Y vbat=3395 | mem=190308
See that almost all processing continues on CPU 0.

If you enable the task (default), see how this gets optimized

Code: Select all

I (0) cpu_start: App cpu up.
I (293)P(05) heap_init: Initializing. RAM available for dynamic allocation:
I (297)P(04) heap_init: At 3FFAFF10 len 000000F0 (0 KiB): DRAM
I (300)P(03) heap_init: At 3FFCA800 len 00015800 (86 KiB): DRAM
I (303)P(03) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (306)P(03) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (309)P(03) heap_init: At 40092770 len 0000D890 (54 KiB): IRAM
I (312)P(03) cpu_start: Pro cpu start user code
I (324)P(12) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (325)P(01) main: (app_main)(C0) Initializing
I (325)P(00) util: (esp32Initialize)(C0) Initalizing Esp32 ...
I (345)P(20) util: (esp32Initialize)(C0) Esp32 initialized
D (345)P(00) peripherals: (peripheralsInitialize)(C0) Initializing peripherals ...
D (345)P(00) peripherals: (gpioInitialize)(C0) Initializing GPIO ...
I (355)P(10) gpio: GPIO[4]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 0| Pulldown: 1| Intr:1 
I (355)P(00) gpio: GPIO[16]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 0| Pulldown: 1| Intr:3 
I (365)P(10) gpio: GPIO[17]| InputEn: 1| OutputEn: 0| OpenDrain: 0| Pullup: 0| Pulldown: 1| Intr:3 
I (365)P(00) gpio: GPIO[5]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0 
I (365)P(00) gpio: GPIO[12]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 1| Intr:0 
D (375)P(10) peripherals: (gpioInitialize)(C0) GPIO initalized
D (375)P(00) peripherals: (adcInitialize)(C0) Initializing ADC ...
D (385)P(10) peripherals: (adcInitialize)(C0) ADC Initialized
D (385)P(00) peripherals: (peripheralsInitialize)(C0) Peripherals initialized
I (385)P(00) ble_server: (initialize)(C0) Initializing BLE Server - device name=Esp32_Device_
I (395)P(10) BTDM_INIT: BT controller compile version [a9156c1]

I (395)P(00) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (455)P(60) phy: phy_version: 3960, 5211945, Jul 18 2018, 10:40:07, 0, 0
I (725)P!(270) ble_uart_server: (ble_uart_server_Initialize)(C0) BLE UART server initialized, device name Esp32_Device_7CCE
I (725)P(00) ble_server: (initialize)(C0) Task for events started
I (725)P(00) ble_server: (events_Task)(C1) Initializing ble events Task
I (735)P(10) ble_server: (initialize)(C0) BLE initialized
I (735)P(00) ble-server: (bleInitialize)(C0) BLE Server initialized!
I (745)P(10) main: (main_Task)(C1) Starting main Task
D (745)P(00) main: (appInitialize)(C1) Initializing ...
D (745)P(00) main: (appInitialize)(C1) Initialized
D (2725)P!(1980) ble_server: (events_Task)(C1) Event received -> 1
I (2725)P(00) ble_server: (processEventConnection)(C1) BLE client connected
D (2725)P(00) ble-server: (onConnect)(C1) Ble connected!
I (2815)P(90) ble_server: (bleCallbackMTU)(C0) BLE MTU changed to 185
V (3655)P!(840) ble_server: (bleCallbackReceiveData)(C0) BLE received [4] : 01:\n
D (3655)P(00) ble_server: (bleCallbackReceiveData)(C0) BLE line message received: 01:
D (3655)P(00) ble_server: (events_Task)(C1) Event received -> 2
V (3655)P(00) main: (processBleMessage)(C1) Code -> 1 Message -> 01:
V (3665)P(10) main: (debugInitial)(C1) Debugging is on now
V (3665)P(00) main: (debugInitial)(C1) Firmware device version: 0.1.0
D (3665)P(00) main: (appInitialize)(C1) Initializing ...
D (3675)P(10) main: (appInitialize)(C1) Initialized
D (3675)P(00) main: (notifyMainTask)(C1) action=1
V (3675)P(00) ble_server: (send)(C1) BLE message [13] -> 01:0.1.0:Y:Y\n
D (3675)P(00) main: (checkEnergyVoltage)(C1) vbat=3395 diff=3395
D (3685)P(10) main: (main_Task)(C1) Notification received -> 1
D (3685)P(00) main: (main_Task)(C1) Reseted time
V (3685)P(00) ble_server: (send)(C1) BLE message [15] -> 10:EXT:Y:3395:\n
V (5095)P!(1410) ble_server: (bleCallbackReceiveData)(C0) BLE received [7] : 11:ALL\n
D (5095)P(00) ble_server: (bleCallbackReceiveData)(C0) BLE line message received: 11:ALL
D (5095)P(00) ble_server: (events_Task)(C1) Event received -> 2
V (5095)P(00) main: (processBleMessage)(C1) Code -> 11 Message -> 11:ALL
V (5105)P(10) main: (sendInfo)(C1) type=ALL
V (5105)P(00) main: (sendInfo)(C1) rom_phy_get_vdd33=6747
D (5105)P(00) main: (checkEnergyVoltage)(C1) vbat=3395 diff=0
V (5105)P(00) ble_server: (send)(C1) BLE message [15] -> 10:EXT:Y:3395:\n
V (5115)P(10) ble_server: (send)(C1) BLE message [237] -> 11:ESP32:*** Chip Info#* Model; 1#* Revision; 1#* Cores; 2#* FreeRTOS unicore ?; No#* ESP-IDF;#  v3.1-beta1-216-gd91c4251-dirty#*** BLE info#* Device name; Esp32_Device_7CCE#* Mac-address; 30;AE;A4;07;7C;CE#\n11:FMEM:184612\n11:VDD33:6747\n
V (5125)P(10) ble_server: (send)(C1) BLE sending part [180] -> 11:ESP32:*** Chip Info#* Model; 1#* Revision; 1#* Cores; 2#* FreeRTOS unicore ?; No#* ESP-IDF;#  v3.1-beta1-216-gd91c4251-dirty#*** BLE info#* Device name; Esp32_Device_7CCE#* Mac- [max=180]
V (5135)P(10) ble_server: (send)(C1) BLE sending part [57] -> address; 30;AE;A4;07;7C;CE#\n11:FMEM:184612\n11:VDD33:6747\n [max=180]
D (8685)P!(3550) main: (main_Task)(C1) * Secs=5 | sensors: vext=Y charging=Y vbat=3393 | mem=184828
Please supplement with more considerations.

Hope this helps.

Who is online

Users browsing this forum: Majestic-12 [Bot], StuartsProjects and 106 guests