SD card interface speed issues

maxwell32
Posts: 5
Joined: Thu Apr 13, 2017 6:14 pm

SD card interface speed issues

Postby maxwell32 » Mon May 22, 2017 12:10 pm

Hello,

i'm working with the SD card interface and I need to acheive a data rate of minimum 2 MByte/s for reading. At the moment I measured only 0.9 MByte/s and I can't get faster. :?

I'm using an Intenso 8 GB Class 10 card in default speed mode with external pullups in Slot 1 and 4 Bit mode. I created a VFS with esp_vfs_fat_sdmmc_mount (as described in the sd card storage example) and read 54080 bytes into a buffer with the fread-function.

Theoretically I could easily achieve more than 2 MByte/s with a Class 10 card. Do you have any ideas what the bottleneck at the ESP32 and the sd card interface is? What is your experience with the data rate of sd cards?

Thank you!

ESP_igrr
Posts: 2067
Joined: Tue Dec 01, 2015 8:37 am

Re: SD card interface speed issues

Postby ESP_igrr » Mon May 22, 2017 12:28 pm

Here are the few benchmarks we did when writing the SDMMC driver.

First are the raw reads and writes — these test the SDMMC peripheral itself by reading/writing certain number of sectors (the number of sectors is given in the "count" column). The tests were run with a no-name SDHC card, which supported Default Speed mode only (25MHz, so ESP32 uses 20MHz clock here).

Code: Select all

 sector  | count | size(kB) | wr_time(ms) | wr_speed(MB/s) | rd_time(ms) | rd_speed(MB/s)
        0 |    1  |    0.5   |     7.45    |      0.07      |    0.43s    |     1.13
        0 |    4  |    2.0   |     3.56    |      0.55      |    0.62s    |     3.18
        1 |   16  |    8.0   |     4.31    |      1.81      |    1.18s    |     6.64
       16 |   32  |   16.0   |     3.99    |      3.92      |    2.24s    |     6.98
       48 |   64  |   32.0   |     6.34    |      4.93      |    3.93s    |     7.96
      128 |  128  |   64.0   |    11.19    |      5.59      |    7.12s    |     8.77
 15366080 |   32  |   16.0   |     4.03    |      3.87      |    2.05s    |     7.61
 15366080 |   64  |   32.0   |     4.48    |      6.98      |    3.74s    |     8.36
 15366136 |    1  |    0.5   |     1.47    |      0.33      |    0.41s    |     1.20
  7683072 |    1  |    0.5   |     4.74    |      0.10      |    0.43s    |     1.13
  7683072 |    4  |    2.0   |     3.37    |      0.58      |    0.63s    |     3.11
  7683072 |    8  |    4.0   |     1.68    |      2.32      |    0.82s    |     4.75
  7683072 |   16  |    8.0   |     2.05    |      3.81      |    1.20s    |     6.49
  7683072 |   32  |   16.0   |     4.73    |      3.31      |    2.05s    |     7.61
  7683072 |   64  |   32.0   |     9.21    |      3.39      |    3.74s    |     8.36
  7683072 |  128  |   64.0   |     9.63    |      6.49      |    7.12s    |     8.78
As you see, you get the highest possible write speed if you can write 64kB of data sequentially.


Next are the benchmarks of FATFS performance. These read and write files on a FAT partition on an SD card.

Code: Select all

Wrote 4194304 bytes (block size 4096) in 7141.901ms (0.560 MB/s)
Wrote 4194304 bytes (block size 8192) in 2133.035ms (1.875 MB/s)
Wrote 4194304 bytes (block size 16384) in 1528.087ms (2.618 MB/s)
Read 4194304 bytes (block size 4096) in 3591.367ms (1.114 MB/s)
Read 4194304 bytes (block size 8192) in 3584.507ms (1.116 MB/s)
Read 4194304 bytes (block size 16384) in 3582.571ms (1.117 MB/s)
As you see, performance is much lower than when writing sectors to the SD card. This is because FATFS is a library optimized for memory use, rather than performance. So it can not buffer the data and coalesce it into contiguous write operations.

This is something you can possibly improve, if you need this, by modifying diskio.c. Essentially you can allocate a 64kB buffer and write sectors coming from FATFS there first. If the next sector is not following the sequence, or the 64kB buffer is full, you write the contents of the buffer to the SD card and start filling it up again. If you have this temporary buffer, you can also make writes asynchronous, i.e. you can tell FATFS that data was written before it is fully written. The application can then go on doing its business (like preparing more data to be written) while DMA in SDMMC driver takes care of the transfer.

Edit: I've read your post again and noticed you are actually asking about reads, not writes.
The story with modifications to diskio.c is pretty much the same, you can try reading more data ahead of time into a temporary buffer, and then returning this data to the application. This obviously comes at the cost of an extra buffer in RAM.

maxwell32
Posts: 5
Joined: Thu Apr 13, 2017 6:14 pm

Re: SD card interface speed issues

Postby maxwell32 » Sat May 27, 2017 9:58 am

Thank you!
I switched to using writing/reading sectors instead of the FATFS library. Now I reached about 9 MB/s for reading 50KB (block size 512 Bytes). I also created my own simple file system that is specific for my application.

turkeylegs
Posts: 6
Joined: Mon Jun 11, 2018 4:11 pm

Re: SD card interface speed issues

Postby turkeylegs » Wed Jul 18, 2018 7:19 pm

Hey, has anybody tried implementing this yet? And what does it look like?

Thanks!

adamtegen
Posts: 4
Joined: Wed Aug 01, 2018 10:12 pm

Re: SD card interface speed issues

Postby adamtegen » Thu Aug 02, 2018 3:36 pm

I have tracked what I think are the performance issues down to: esp_vfs_read() is being called to read 128 bytes at a time, even when I am calling an fread(), requesting 50KB. I can't find where that 50KB request is being broken into 128 byte chunks. I assume its somewhere in newlib or the configuration/setup of newlib, but I can't find an evidence of that.

Any help?

Adam

Milad_f1
Posts: 3
Joined: Tue Sep 11, 2018 9:38 am

Re: SD card interface speed issues

Postby Milad_f1 » Wed Oct 03, 2018 1:53 am

adamtegen wrote:I have tracked what I think are the performance issues down to: esp_vfs_read() is being called to read 128 bytes at a time, even when I am calling an fread(), requesting 50KB. I can't find where that 50KB request is being broken into 128 byte chunks. I assume its somewhere in newlib or the configuration/setup of newlib, but I can't find an evidence of that.

Any help?

Adam
Hi Adam,

Have you found any solution for the problem? I have the same issue. I can write into SD card with the speed of ~6MB using fwrite but the reading speed from the SD card (fread) is aroung 1MB!!

snahmad75
Posts: 445
Joined: Wed Jan 24, 2018 6:32 pm

Re: SD card interface speed issues

Postby snahmad75 » Thu Feb 07, 2019 9:19 pm

Hi,

Anyone found issue why it is slow.

esp_vfs_read() is being called to read 128 bytes at a time
Can we speed up.

snahmad75
Posts: 445
Joined: Wed Jan 24, 2018 6:32 pm

Re: SD card interface speed issues

Postby snahmad75 » Fri Mar 15, 2019 4:00 pm

My SD card read is better now. I change mongoose web server which serve files. changes it stack buffer size from 1k to 4K
Better speed now.

Still puzzles with these settings.
allocation_unit_size = 64K

Our SD card is by default formatted with 32K. Is this issue?



// If format_if_mount_failed is set to true, SD card will be partitioned and
// formatted in case when mounting fails.
esp_vfs_fat_sdmmc_mount_config_t mount_config = {
.format_if_mount_failed = true,
.max_files = 5,
.allocation_unit_size = 64 * 1024
};

// Use settings defined above to initialize SD card and mount FAT filesystem.
// Note: esp_vfs_fat_sdmmc_mount is an all-in-one convenience function.
// Please check its source code and implement error recovery when developing
// production applications.
sdmmc_card_t* card;
esp_err_t ret = esp_vfs_fat_sdmmc_mount("/sdcard", &host, &slot_config, &mount_config, &card);

Who is online

Users browsing this forum: spenderIng and 163 guests