Commerce is a highly flexible and scalable product containing solutions for merchants of all sizes. This section covers best practices and recommendations on configuring Commerce to work with large amounts of data, extreme load, and other enterprise cases.
Calibrate index performance
When dealing with large amounts of data, re-indexing can become a concern. The Commerce team selected the most loaded indexes and enabled batch indexing, which allows you to set a portion of data to be processed on each iteration. This way, the user can tune batch sizes based on the type and size of data in the database.
To manage this setting, edit the batchRowsCount parameter in the di.xml file of the corresponding module. The following indexes support this feature:
Category Product Index (Catalog Module)
Price Index (Catalog Module)
EAV Index (Catalog Module)
Stock Index (CatalogInventory Module)
You can tune indexer performance by adjusting the index batching size variables. This controls how many entities are processed at a time by the indexer. In some situations, we have seen significant decreases in indexing time.
For example, if you are running a profile similar to B2B Medium, you can override the configuration value batchRowsCount in app/code/Magento/catalog/etc/di.xml and override the default value of 5000 to 1000. This reduces the full indexing time from 4 hours down to 2 hours with a default MySQL configuration.
NOTE
We have not enabled batching for the catalog rules indexer. Merchants with a large number of catalog rules need to adjust their MySQL configuration to optimize indexing time. In this case, editing your MySQL configuration file and allocating more memory to the TMP_TABLE_SIZE and MAX_HEAP_TABLE_SIZE configuration values (the default is 16M for both) will improve performance for this indexer, but will result in MySQL consuming more RAM.
Limit customer groups and shared catalogs by websites
A large number of product SKUs, websites, customer groups, or shared catalogs will impact the running time of the Product Price and Catalog Rule indexers. This is because by default, all websites are assigned to all customer groups (shared catalogs).
To decrease indexation time, you can exclude certain websites from customer groups (shared catalogs).
Set up Redis
Sometimes one Redis instance is not enough to serve incoming requests. There are several solutions that we can recommend to address this situation.
First, Commerce allows you to configure separate cache storage for each cache type. This allows you to install as many separate Redis instances as the number of types of cache that are registered in Magento. Realistically, you might want Redis instances for the most actively used caches, such as configuration, layout, and blocks.
Another solution can be to place the configuration cache on the file system and move the other caches to the Redis server. With this solution, you need a separate tool for centralized invalidation of configuration cache on all your web nodes.
You could also use a Redis cluster that performs parallel read/write operations with an automatically-increasing number of nodes. Commerce does not provide configuration for this case, but it can be launched with minor customizations.
Set up RabbitMQ
Adobe Commerce supports message queues implemented through RabbitMQ. Commerce uses this service for executing numerous asynchronous operations, including B2B catalog operations and asynchronous stock updates. All interfaces for adding more jobs to the job server are distributed with the product and are available for custom asynchronous logic implementation in the scope of third-party extensions. As with any other integration, Commerce provides a sample configuration file for RabbitMQ that contains all recommended settings and is fully ready for production usage.
Split the database
WARNING
The split database feature was deprecated in version 2.4.2 of Adobe Commerce. See Revert from a split database to a single database.
Adobe Commerce allows you to configure scalable database storage to meet the needs of a growing business. You can set up three separate master databases that serve specific domains:
Main (Catalog) Database
Checkout Database
Order Management System (OMS) Database
To configure additional databases, you must create an empty database and run one of the following commands:
Calibrate index performance
When dealing with large amounts of data, re-indexing can become a concern. The Commerce team selected the most loaded indexes and enabled batch indexing, which allows you to set a portion of data to be processed on each iteration. This way, the user can tune batch sizes based on the type and size of data in the database.
To manage this setting, edit the batchRowsCount parameter in the di.xml file of the corresponding module. The following indexes support this feature:
Category Product Index (Catalog Module)
Price Index (Catalog Module)
EAV Index (Catalog Module)
Stock Index (CatalogInventory Module)
You can tune indexer performance by adjusting the index batching size variables. This controls how many entities are processed at a time by the indexer. In some situations, we have seen significant decreases in indexing time.
For example, if you are running a profile similar to B2B Medium, you can override the configuration value batchRowsCount in app/code/Magento/catalog/etc/di.xml and override the default value of 5000 to 1000. This reduces the full indexing time from 4 hours down to 2 hours with a default MySQL configuration.
NOTE
We have not enabled batching for the catalog rules indexer. Merchants with a large number of catalog rules need to adjust their MySQL configuration to optimize indexing time. In this case, editing your MySQL configuration file and allocating more memory to the TMP_TABLE_SIZE and MAX_HEAP_TABLE_SIZE configuration values (the default is 16M for both) will improve performance for this indexer, but will result in MySQL consuming more RAM.
Limit customer groups and shared catalogs by websites
A large number of product SKUs, websites, customer groups, or shared catalogs will impact the running time of the Product Price and Catalog Rule indexers. This is because by default, all websites are assigned to all customer groups (shared catalogs).
To decrease indexation time, you can exclude certain websites from customer groups (shared catalogs).
Set up Redis
Sometimes one Redis instance is not enough to serve incoming requests. There are several solutions that we can recommend to address this situation.
First, Commerce allows you to configure separate cache storage for each cache type. This allows you to install as many separate Redis instances as the number of types of cache that are registered in Magento. Realistically, you might want Redis instances for the most actively used caches, such as configuration, layout, and blocks.
Another solution can be to place the configuration cache on the file system and move the other caches to the Redis server. With this solution, you need a separate tool for centralized invalidation of configuration cache on all your web nodes.
You could also use a Redis cluster that performs parallel read/write operations with an automatically-increasing number of nodes. Commerce does not provide configuration for this case, but it can be launched with minor customizations.
Set up RabbitMQ
Adobe Commerce supports message queues implemented through RabbitMQ. Commerce uses this service for executing numerous asynchronous operations, including B2B catalog operations and asynchronous stock updates. All interfaces for adding more jobs to the job server are distributed with the product and are available for custom asynchronous logic implementation in the scope of third-party extensions. As with any other integration, Commerce provides a sample configuration file for RabbitMQ that contains all recommended settings and is fully ready for production usage.
Split the database
WARNING
The split database feature was deprecated in version 2.4.2 of Adobe Commerce. See Revert from a split database to a single database.
Adobe Commerce allows you to configure scalable database storage to meet the needs of a growing business. You can set up three separate master databases that serve specific domains:
Main (Catalog) Database
Checkout Database
Order Management System (OMS) Database
To configure additional databases, you must create an empty database and run one of the following commands: