Page 1 of 1

SD card interface speed issues

Posted: Mon May 22, 2017 12:10 pm
by maxwell32
Hello,

i'm working with the SD card interface and I need to acheive a data rate of minimum 2 MByte/s for reading. At the moment I measured only 0.9 MByte/s and I can't get faster. :?

I'm using an Intenso 8 GB Class 10 card in default speed mode with external pullups in Slot 1 and 4 Bit mode. I created a VFS with esp_vfs_fat_sdmmc_mount (as described in the sd card storage example) and read 54080 bytes into a buffer with the fread-function.

Theoretically I could easily achieve more than 2 MByte/s with a Class 10 card. Do you have any ideas what the bottleneck at the ESP32 and the sd card interface is? What is your experience with the data rate of sd cards?

Thank you!

Re: SD card interface speed issues

Posted: Mon May 22, 2017 12:28 pm
by ESP_igrr
Here are the few benchmarks we did when writing the SDMMC driver.

First are the raw reads and writes — these test the SDMMC peripheral itself by reading/writing certain number of sectors (the number of sectors is given in the "count" column). The tests were run with a no-name SDHC card, which supported Default Speed mode only (25MHz, so ESP32 uses 20MHz clock here).

Code: Select all

 sector  | count | size(kB) | wr_time(ms) | wr_speed(MB/s) | rd_time(ms) | rd_speed(MB/s)
        0 |    1  |    0.5   |     7.45    |      0.07      |    0.43s    |     1.13
        0 |    4  |    2.0   |     3.56    |      0.55      |    0.62s    |     3.18
        1 |   16  |    8.0   |     4.31    |      1.81      |    1.18s    |     6.64
       16 |   32  |   16.0   |     3.99    |      3.92      |    2.24s    |     6.98
       48 |   64  |   32.0   |     6.34    |      4.93      |    3.93s    |     7.96
      128 |  128  |   64.0   |    11.19    |      5.59      |    7.12s    |     8.77
 15366080 |   32  |   16.0   |     4.03    |      3.87      |    2.05s    |     7.61
 15366080 |   64  |   32.0   |     4.48    |      6.98      |    3.74s    |     8.36
 15366136 |    1  |    0.5   |     1.47    |      0.33      |    0.41s    |     1.20
  7683072 |    1  |    0.5   |     4.74    |      0.10      |    0.43s    |     1.13
  7683072 |    4  |    2.0   |     3.37    |      0.58      |    0.63s    |     3.11
  7683072 |    8  |    4.0   |     1.68    |      2.32      |    0.82s    |     4.75
  7683072 |   16  |    8.0   |     2.05    |      3.81      |    1.20s    |     6.49
  7683072 |   32  |   16.0   |     4.73    |      3.31      |    2.05s    |     7.61
  7683072 |   64  |   32.0   |     9.21    |      3.39      |    3.74s    |     8.36
  7683072 |  128  |   64.0   |     9.63    |      6.49      |    7.12s    |     8.78
As you see, you get the highest possible write speed if you can write 64kB of data sequentially.


Next are the benchmarks of FATFS performance. These read and write files on a FAT partition on an SD card.

Code: Select all

Wrote 4194304 bytes (block size 4096) in 7141.901ms (0.560 MB/s)
Wrote 4194304 bytes (block size 8192) in 2133.035ms (1.875 MB/s)
Wrote 4194304 bytes (block size 16384) in 1528.087ms (2.618 MB/s)
Read 4194304 bytes (block size 4096) in 3591.367ms (1.114 MB/s)
Read 4194304 bytes (block size 8192) in 3584.507ms (1.116 MB/s)
Read 4194304 bytes (block size 16384) in 3582.571ms (1.117 MB/s)
As you see, performance is much lower than when writing sectors to the SD card. This is because FATFS is a library optimized for memory use, rather than performance. So it can not buffer the data and coalesce it into contiguous write operations.

This is something you can possibly improve, if you need this, by modifying diskio.c. Essentially you can allocate a 64kB buffer and write sectors coming from FATFS there first. If the next sector is not following the sequence, or the 64kB buffer is full, you write the contents of the buffer to the SD card and start filling it up again. If you have this temporary buffer, you can also make writes asynchronous, i.e. you can tell FATFS that data was written before it is fully written. The application can then go on doing its business (like preparing more data to be written) while DMA in SDMMC driver takes care of the transfer.

Edit: I've read your post again and noticed you are actually asking about reads, not writes.
The story with modifications to diskio.c is pretty much the same, you can try reading more data ahead of time into a temporary buffer, and then returning this data to the application. This obviously comes at the cost of an extra buffer in RAM.

Re: SD card interface speed issues

Posted: Sat May 27, 2017 9:58 am
by maxwell32
Thank you!
I switched to using writing/reading sectors instead of the FATFS library. Now I reached about 9 MB/s for reading 50KB (block size 512 Bytes). I also created my own simple file system that is specific for my application.

Re: SD card interface speed issues

Posted: Wed Jul 18, 2018 7:19 pm
by turkeylegs
Hey, has anybody tried implementing this yet? And what does it look like?

Thanks!

Re: SD card interface speed issues

Posted: Thu Aug 02, 2018 3:36 pm
by adamtegen
I have tracked what I think are the performance issues down to: esp_vfs_read() is being called to read 128 bytes at a time, even when I am calling an fread(), requesting 50KB. I can't find where that 50KB request is being broken into 128 byte chunks. I assume its somewhere in newlib or the configuration/setup of newlib, but I can't find an evidence of that.

Any help?

Adam

Re: SD card interface speed issues

Posted: Wed Oct 03, 2018 1:53 am
by Milad_f1
adamtegen wrote:I have tracked what I think are the performance issues down to: esp_vfs_read() is being called to read 128 bytes at a time, even when I am calling an fread(), requesting 50KB. I can't find where that 50KB request is being broken into 128 byte chunks. I assume its somewhere in newlib or the configuration/setup of newlib, but I can't find an evidence of that.

Any help?

Adam
Hi Adam,

Have you found any solution for the problem? I have the same issue. I can write into SD card with the speed of ~6MB using fwrite but the reading speed from the SD card (fread) is aroung 1MB!!

Re: SD card interface speed issues

Posted: Thu Feb 07, 2019 9:19 pm
by snahmad75
Hi,

Anyone found issue why it is slow.

esp_vfs_read() is being called to read 128 bytes at a time
Can we speed up.

Re: SD card interface speed issues

Posted: Fri Mar 15, 2019 4:00 pm
by snahmad75
My SD card read is better now. I change mongoose web server which serve files. changes it stack buffer size from 1k to 4K
Better speed now.

Still puzzles with these settings.
allocation_unit_size = 64K

Our SD card is by default formatted with 32K. Is this issue?



// If format_if_mount_failed is set to true, SD card will be partitioned and
// formatted in case when mounting fails.
esp_vfs_fat_sdmmc_mount_config_t mount_config = {
.format_if_mount_failed = true,
.max_files = 5,
.allocation_unit_size = 64 * 1024
};

// Use settings defined above to initialize SD card and mount FAT filesystem.
// Note: esp_vfs_fat_sdmmc_mount is an all-in-one convenience function.
// Please check its source code and implement error recovery when developing
// production applications.
sdmmc_card_t* card;
esp_err_t ret = esp_vfs_fat_sdmmc_mount("/sdcard", &host, &slot_config, &mount_config, &card);