4.5.4. eMMC Stress Test

4.5.4.1. Test Principle

The eMMC stress test uses the iozone tool to perform stress testing on the eMMC storage device, simulating various I/O operation patterns and measuring performance indicators such as read/write speed, throughput, and latency. The specific test principles are as follows:

  • Create Test Files: IOzone first creates one or more files on the eMMC storage. The size and number of these files can be customized to simulate different types of workloads. During file creation, IOzone uses a specific block size for read and write operations.

  • Execute Multiple I/O Operations:

    • Sequential Write: Continuously write data to the storage device to test its maximum write bandwidth.

    • Sequential Read: Continuously read data to test the device’s read bandwidth.

    • Random Write: Perform random writes to test the device’s random write capability.

    • Random Read: Perform random reads to test the device’s random read capability.

    • Mixed Read/Write: Simulate multiple read/write operations to test the device’s overall performance.

    • Re-write: Modify or overwrite existing files to test the device’s performance when updating files.

  • Data Recording and Analysis: IOzone records the execution time of each operation and calculates key metrics such as throughput and latency to generate data reports.

Test Content

The eMMC stress test includes two test scripts: emmc_performance_test.sh and emmc_stability_test.sh, representing eMMC performance testing and eMMC stability stress testing, respectively:

eMMC Stability Stress Test

1. Test Objective:

  • Stability Stress Test Script: The main purpose is to verify system stability through long-duration stress testing. Typically, the -z (file fill) option and large file settings (e.g., -n and -g parameters) are used to ensure reliability verification under prolonged operation.

2. Command Explanation:

  • Stability Test: iozone -e -I -az -n 16m -g 2g -q 16m -f "$output_dir/iozone_data" -Rb "$output_dir/test_iozone_emmc_ext4_stability_${loop_num}.xls"

  • Parameter Explanation:

    • -e: Enable extended test mode, allowing IOzone to perform additional test types such as rewrite, reverse write, and EOF write.

    • -I: Enable direct I/O (O_DIRECT), bypassing the operating system cache for direct disk read/write.

    • -a: Run in auto mode, where IOzone automatically tests various operations and file sizes, including sequential and random read/write.

    • -z: In this command, used to fill files during each test to ensure continuous read/write operations, increasing system stress—commonly used in stability tests.

    • -n 16m: Set the minimum data block size for write tests to 16MB, meaning testing starts at 16MB and gradually increases.

    • -g 2g: Set the maximum test file size to 2GB, indicating IOzone will test read/write performance for files up to 2GB.

    • -q 16m: Set the file block size used by IOzone to 16MB, which also defines the memory buffer size.

    • -f "$output_dir/iozone_data": Specify the path where test results are stored.

    • -Rb "$output_dir/test_iozone_emmc_ext4_stability_${loop_num}.xls": Output test results in Excel (.xls) format.

eMMC Performance Test

1. Test Objective:

  • Performance Test Script: Primarily used to measure storage device performance under different file and record sizes. The test evaluates performance across various scenarios using combinations of multiple record sizes (-r) and file sizes (-s).

2. Command Explanation:

  • Performance Test: iozone -e -I -a -r 4K -r 16K -r 64K -r 256K -r 1M -r 4M -r 16M -s 16K -s 1M -s 16M -s 128M -s 256M -f "$output_dir/iozone_data" -Rb "$output_dir/test_iozone_emmc_ext4_performance_${loop_num}.xls"

  • Parameter Explanation:

    • -e: Enable extended test.

    • -I: Enable direct I/O, bypassing OS cache.

    • -a: Run in auto mode, covering multiple read/write operations with various file and record sizes.

    • -r 4K -r 16K -r 64K -r 256K -r 1M -r 4M -r 16M: Specify record sizes used in testing, affecting eMMC performance across different data block dimensions.

    • -s 16K -s 1M -s 16M -s 128M -s 256M: Define test file size ranges—from 16KB to 256MB—to evaluate read/write performance across different file sizes.

    • -f "$output_dir/iozone_data": Specify the storage location for test files.

    • -Rb "$output_dir/test_iozone_emmc_ext4_performance_${loop_num}.xls": Output test results in Excel format.

4.5.4.2. Preparation

1. Run the lsblk -f command to ensure the eMMC storage device is properly mounted and has sufficient space for testing.

root@buildroot:/userdata/gpu2d_test# lsblk -f
NAME         FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
mtdblock0
mmcblk0
|-mmcblk0p1
|-mmcblk0p2
|-mmcblk0p3
|-mmcblk0p4
|-mmcblk0p5
|-mmcblk0p6
|-mmcblk0p7
|-mmcblk0p8
|-mmcblk0p9  ext4   1.0   /     de7aed37-7c14-4ace-b0bf-2b68a2f9ed7b       0   100% /
|-mmcblk0p10 ext4   1.0   hbre  c0e42531-9464-4908-a53f-d2e753512238   78.8M    51% /usr/hobot
|-mmcblk0p11 ext4   1.0   app   420a2fb6-82e2-47bf-b126-6b95b948ed1c  495.6M    23% /app
`-mmcblk0p12 ext4   1.0   cache 4f1aa72c-2d55-4d56-87e1-e82017e426e4    4.7G    10% /userdata
mmcblk0boot0
mmcblk0boot1

In the eMMC performance test, the command -s 256MB creates a file of specified size for read/write testing, e.g., -s 256M -f "$output_dir/iozone_data". Please ensure the available space under the /app mount point meets the requirements for maximum file read/write testing.

2. Confirm that the two test scripts emmc_performance_test.sh and emmc_stability_test.sh exist in the path /app/platform_samples/chip_base_test/02_emmc/.

02_emmc/
├── emmc_performance_test.sh
└── emmc_stability_test.sh

4.5.4.3. Test Method

The stress test scripts support the -h suffix to display command parameter descriptions (applicable to both scripts):

./emmc_performance_test.sh -h

Usage: ./emmc_performance_test.sh [options]

Options:
  -t <time>       Set the test duration (e.g., 2h for hours, 30m for minutes; default: 48h).
  -d <seconds>    Set the sleep time between loops in seconds (default: 30).
  -o <directory>  Set the output directory for logs (default: script's '../output' folder).
  -h              Show this help message and exit.

Parameter explanations:

  • -t <time>: Set test duration, e.g., 2h means 2 hours, 30m means 30 minutes; default is 48 hours.

  • -d <seconds>: Set sleep time between loops in seconds; default is 30 seconds.

  • -o <directory>: Set log output directory; default is the ../output folder relative to the script location.

  • -h: Display help message and exit.

Example: For example, using the command: ./emmc_performance_test.sh -t 2h -d 10 -o /userdata/output sets a custom test duration of 2 hours, loop interval of 10 seconds, and output directory as /userdata/output.

eMMC Stability Test:

After completing preparation, run the test command:

app/platform_samples/chip_base_test/02_emmc/

./emmc_stability_test.sh

After running for a period, output appears as follows:

eMMC stability test starting...
Test configuration:
  Test duration: 2880 minutes
  Sleep duration: 30 seconds
  Output directory: /app/platform_samples/chip_base_test/log
loop_test: 1
        Iozone: Performance Test of File I/O
                Version $Revision: 3.493 $
                Compiled for 64 bit mode.
                Build: linux

        Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                     Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                     Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                     Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                     Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
                     Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
                     Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
                     Vangel Bojaxhi, Ben England, Vikentsi Lapa,
                     Alexey Skidanov, Sudhir Kumar.

        Run began: Thu Jan  1 03:53:27 1970

        Include fsync in write timing
        O_DIRECT feature enabled
        Auto Mode
        Cross over of record size disabled.
        Using minimum file size of 16384 kilobytes.
        Using maximum file size of 2097152 kilobytes.
        Using Maximum Record Size 16384 kB
        Excel chart generation enabled
        Command line used: iozone -e -I -az -n 16m -g 2g -q 16m -f /app/platform_samples/chip_base_test/log/iozone_data -Rb /app/platform_samples/chip_base_test/log/test_iozone_emmc_stability_1.xls
        Output is in kBytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 kBytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
           16384       4    16528    18226    24101    23748    24132    27420    21336     22977     21307    74257    73024   746423   508393
           16384       8    23396    25740    26597    26515    22837    28021    21191     33722     21453    70527    78517   501653   507889
           16384      16    32617    36313    45886    45603    39386    37817    37385     52995     37303    73691    75478  1155837   823647
           16384      32    49805    56460    58152    58100    54770    27146    51481     27408     61609    82317    70461   798713   767005
           16384      64    62190    60412    97321   104999    81189    36860    82208     37939     92583    81828    79986  1130242   642484
           16384     128    77980    76682   133045   135420   118054    61150   110984     56217    121902    78314    76628   856053   832562
           16384     256    75600    96903   151310   154328   141069    74830   114178     67056    142496    81797    74447   712936   496740
           16384     512    84511    95697   140845   144154   158989    90900   153428     82660    157005    82618    73102   870147   849219
           16384    1024    89460    96806   148825   159890   163213    48185   144125     94222    143921    62020    87725  1553869   827098
           16384    2048    99009    90244   151417   155064   153802    96864   151189     98534    149906    61780    86793   659528   521202
           16384    4096   100021    90433   156428   159981   159845    98881   155403    100275    156243    65415    73284  1298977   878545
           16384    8192   101815   102025   158681   161991   161900    91054   155868    101216    158402    64583    67735   607062   815936
           16384   16384    99572   103136   161970   164727   165066   102090   162238     91487    160470    72264    74101   611891   826056
           32768       4    18622    21099    25151    25152    25244    28032    21607     25818     21371    75887    82629   508259   856455
           32768       8    26335    30219    30169    30249    24746    30948    23299     41141     23233    76726    86830  1665040  1162150
           32768      16    36097    39610    47906    48403    40690    41694    37340     58452     36658    77680    87567   847682   960966
           32768      32    51848    48383    77901    82475    66412    27517    51665     27602     60524    79078    84456  1582842  1041245
           32768      64    58908    61008    98257   106892    88301    38228    83961     38238     70671    78600    87072   984526   882972
           32768     128    82441    84594   141841   142600   124474    58406   121304     61327    121530    78985    85211  2110116  1571606
           32768     256    94575    92323   156299   157396   145970    70771   143510     78336    144694    78374    85049  2113519  1560826
           32768     512    97020    92220   140765   164216   158572    91175   154818     86774    158036    35928    29897  1596103  1137501
           32768    1024    31191    31041   149915   158399   164417    27155   140115     28060    142770    25172    26510  1741589  1375131
           32768    2048    26899    27783   147326   148335   146382    28053   153602     27722    138546    25156    29453  1706311  1320332
           32768    4096    31098    30301   154606   156154   156925    30463   152405     30380    156005    27478    28423  1633662  1319640
           32768    8192    30679    30385   158022   159438   159458    29472   154650     30223    151013    27118    28941  1644072  1314189
           32768   16384    29761    30626   162190   163566   163254    30455   161151     29771    154181    27460    29196  1614823  1306695

Key Information Explanation:

  • Test duration: Test duration in minutes (i.e., 48 hours).

  • Sleep duration: Sleep time between loops—30 seconds—to allow system and storage recovery, reducing fluctuations.

  • Output directory: /app/platform_samples/chip_base_test/log

  • Command line used: iozone -e -I -az -n 16m -g 2g -q 16m -f /app/platform_samples/chip_base_test/log/iozone_data -Rb /app/platform_samples/chip_base_test/log/test_iozone_emmc_stability_1.xls, with key parameters:

    • File size range: minimum 16MB, maximum 2GB.

    • Record size range: from 4KB to 16384KB (16MB).

  • Main Performance Metrics:

    • kB: File size in KB.

    • reclen: Record size in bytes—the data block size per I/O operation.

    • random write / rewrite / read / reread: Throughput during random write, rewrite, random read, and re-read.

    • bkwd (Backward Read): Throughput for backward (reverse) read.

    • record write / record read: Throughput for sequential write and read.

    • stride read / stride write: Throughput for stride (jumped) read and write.

    • fwrite / frewrite / fread / freread: Throughput for direct I/O operations using O_DIRECT flag.

eMMC Performance Test:

After completing preparation, run the test command:

app/platform_samples/chip_base_test/02_emmc/

./emmc_performance_test.sh

After running for a period, output appears as follows:

Output is in kBytes/sec
        Time Resolution = 0.000001 seconds.
        Processor cache size set to 1024 kBytes.
        Processor cache line size set to 32 bytes.
        File stride size set to 17 * record size.
                                                              random    random     bkwd    record    stride
              kB  reclen    write  rewrite    read    reread    read     write     read   rewrite      read   fwrite frewrite    fread  freread
              16       4     3780     8167      460    12500    12279     5310     4857      7633      7011     7194     8998   727272  1230769
              16      16     6269    17718     9122    41884    38095     6619    28021     11527      8639     6090    17524  1230769  1333333
            1024       4    17325    19623    18309    20130    18949    26827    18673     22446     18655    59413    62814  1519287  1214709
            1024      16    31250    36644    39791    41593    37498    38246    35873     42816     36268    62488    26864   957904   699931
            1024      64    57270    60937    86166    95961    83955    43208    72418     42219     76389    60685    66009  2160337  1802816
            1024     256    84342    87097   122341   149227   140659    79006   112515     72660    117068    28331    72117  1914018  1532934
            1024    1024    91690    93953   133664   161641   162720    98594   134013     90611    133229    75756    84838  1796491  1484057
           16384       4    18423    21648    23625    23736    23659    28863    20940     25469     20881    76403    69075   518645   641302
           16384      16    36059    40422    43281    43681    37900    38980    39529     57907     39596    79792    80306  2223971  1079528
           16384      64    66766    75152   116098   117609    93394    40465    91837     42408     94044    75202    68624  2080243  1789036
           16384     256    91077    88937   152283   159162   141655    67446   138908     77983    143531    82034    74734  1231879   637435
           16384    1024   102095    99983   163597   169954   164941    84258   159818     98017    160611    81357    75563  1996344   853911
           16384    4096   102216   101084   161169   160300   162896   100551   158995     91289    158698    75250    77551  1345818  1381333
           16384   16384    93026   103119   157747   160792   167624   101833   163279     92495    165471    76821    76778  1279300   861046
          131072       4    18723    21457    25046    25080    24999    27855    22198     25599     21949    87720    87353  1363827  1188807
          131072      16    37955    40133    48228    48432    36506    38149    38223     59057     37771    90659    91102  2044709  1630938
          131072      64    58971    69843   108532   108846    85083    37930    94236     39192     91901    90835    91374  2211625  1812790
          131072     256    93938    96103   157272   158124   142723    69904   144578     72571    139679    92895    90228  2196504  1819635
          131072    1024    97717    92395   168437   169027   164048    89288   164161     94035    162668    91980    93660  1742978  1506765
          131072    4096   100309    86737   159881   159958   159632    42273   157815     30580    158251    29852    29923  1383725  1487679
          131072   16384    31068    30734   161342   161741   162055    30712   160999     30645    157931    30094    30414  1662000  1493987
          262144       4    16202    17899    24111    24117    21117    26843    22901     25761     22613    91624    91576  1391777  1192387
          262144      16    36265    39895    48418    48431    35809    38468    41636     58951     41142    93458    84658  2084213  1674945
          262144      64    57271    72327   109199   109366    83766    36119    87882     38508     91967    93643    92303  2137804  1803623
          262144     256    60192    90587   156266   156170   140349    66190   145055     72361    140016    89305    91224  2224254  1826685
          262144    1024    69923    97061   167924   168228   163411    87766   157521     93416    162937    94083    92071  1850332  1568569
          262144    4096    70257    99365   158394   158507   157501    96434   158837     99824    158364    93752    91331  1633662  1470752
          262144   16384    69600   100196   162300   162584   162611    99225   162466    100701    164261    92946    91611  1671452  1471487

Key Information Explanation:

  • Test duration: Test duration in minutes (i.e., 48 hours).

  • Sleep duration: Sleep time between loops—30 seconds—for system and storage recovery.

  • Output directory: /app/platform_samples/chip_base_test/log

  • Command line used: iozone -e -I -a -r 4K -r 16K -r 64K -r 256K -r 1M -r 4M -r 16M -s 16K -s 1M -s 16M -s 128M -s 256M -f "$output_dir/iozone_data" -Rb "$output_dir/test_iozone_emmc_ext4_performance_${loop_num}.xls"

    • Record Sizes: 4KB, 16KB, 64KB, 256KB, 1MB, 4MB, 16MB.

    • File Sizes: 16KB, 1MB, 16MB, 128MB, 256MB.

  • Main Performance Metrics:

    • kB: File size in KB.

    • reclen: Record size in bytes—data block size per I/O operation.

    • random write / rewrite / read / reread: Throughput during random write, rewrite, random read, and re-read.

    • bkwd (Backward Read): Throughput for backward read.

    • record write / record read: Throughput for sequential write and read.

    • stride read / stride write: Throughput for stride (jumped) read and write.

    • fwrite / frewrite / fread / freread: Throughput for direct I/O operations using O_DIRECT.

4.5.4.4. Test Metrics

eMMC Stability Test

After the test program starts, the stability test generates the following files in /app/platform_samples/chip_base_test/log:

  • test_iozone_emmc_stability.log: Logs status information during stress testing.

  • test_iozone_emmc_stability_*.xls: Records data results from stress testing.

The goal is to ensure the system runs stably for 48 hours without rebooting or hanging. To verify stability, check logs for anomalies such as “fail”, “error”, or “timeout”:

cd "/app/platform_samples/chip_base_test/log/" && grep -iE 'error|fail|timeout' test_iozone_emmc_stability*.log

eMMC Stability Test Results

After 24 hours of testing, no abnormal messages were found in the log, indicating the stability stress test passed.

Test loop 1 succeeded!
Test loop 2 succeeded!
Test loop 3 succeeded!
.....

eMMC Performance Test

After the test program starts, the performance test generates the following files in /app/platform_samples/chip_base_test/output:

  • test_iozone_emmc_performance.log: Logs status information during stress testing.

  • test_iozone_emmc_performance_*.xls: Records data results from stress testing.

The goal is to ensure stable operation for 48 hours without reboot or hang. Check logs for “fail”, “error”, or “timeout”:

cd "/app/platform_samples/chip_base_test/output/" && grep -iE 'error|fail|timeout' test_iozone_emmc_performance*.log

Additionally, performance should meet general usage standards. For evb_x5 (eMMC 5.1), the maximum supported mode is HS200. Typically, read speeds range from 90 MB/s to 150 MB/s, and write speeds are slightly lower, between 60 MB/s and 120 MB/s.

eMMC Performance Test Results

After 48 hours of testing, no anomalies were found in the logs. Maximum read speed reached approximately 168 MB/s, and maximum write speed reached approximately 102 MB/s—indicating the performance stress test passed.

Test loop 1 succeeded!
Test loop 2 succeeded!
Test loop 3 succeeded!
.....