Neural networks, a fascinating technology inspired by the human brain, form the basis of artificial intelligence. These networks consist of layers of interconnected nodes, or artificial neurons, that learn patterns from data and make predictions. For example, large language models generate text by predicting the next word or phrase based on the words that came before it.

Traditional neural networks require many internal connections to be adjusted before the model can make accurate predictions. An approach called echo state networks (ESNs) simplifies this process by keeping most of the network fixed and training only the output layer. This reduces the number of parameters to be optimized, speeding up training while lowering computational demands. However, ESN performance is highly sensitive to hyperparameters, the settings that determine how the model behaves. Despite the importance of these hyperparameters, the relationship between the target system and the hyperparameters remains unexplored.

In a study that was published in Volume 17, Issue 3 of Nonlinear Theory and Its Applications (NOLTA), IEICE on July 01, 2026, a research group led by Professor Tohru Ikeguchi and Assistant Professor Kazuya Sawada from the Faculty of Engineering, Tokyo University of Science, Japan, found that these settings can be optimized by accounting for the time scale of the target system, or how quickly it changes over time.

"ESNs are a type of reservoir computing that has attracted attention in recent years. This method is known to be effective for tasks such as time series prediction, but the prediction accuracy depends on how the parameters are set, so it is necessary to set them properly. The novelty of this paper lies in its focus on the time scale of the time series to be predicted when setting these hyperparameters. Our analysis offers a unique perspective that has not been widely discussed in previous literature," says Prof. Ikeguchi.

Previous studies have suggested that the optimal hyperparameters depend on the characteristics of the target system, particularly how patterns change over time, but the relationship has remained unclear. The researchers therefore focused on the time scale, which describes how quickly a system changes over time. They investigated whether systems with similar time scales would also show similar patterns in the hyperparameter settings associated with accurate prediction.

To test this idea, the research team studied three chaotic systems: the Lorenz system, the Rössler system, and the Chua circuit. They adjusted the time scales of these systems using decorrelation time, a measure of how quickly a system becomes less related to its past behavior. The researchers then tested how accurately ESNs predicted these systems across many hyperparameter settings. Figure 1 illustrates the procedure used to match the time scales of the different target systems.

The team also compared two training conditions: one using the same number of training data points and another using the same total trajectory length across different time scales. This allowed them to separate the effects of the time scale from the amount of training data.

When the time scales of the different systems were matched, the ranges of hyperparameter settings linked to high prediction accuracy showed similar patterns across all systems. This suggests that prediction performance may depend more on how a system changes over time than on the specific system itself.

In particular, systems with longer time scales achieved better prediction accuracy when larger spectral radius values—a setting that affects how long information remains in the network—were used. The researchers found that the optimal spectral radius increased as the time scale became longer, as shown in Figure 2.

The findings suggest that the time scale of a target time series may provide a practical way to choose ESN hyperparameters by narrowing the range of appropriate hyperparameters, thereby reducing the need for time-consuming trial-and-error searches. The results also suggest that decorrelation time could help estimate suitable settings directly, potentially improving ESNs in applications such as weather forecasting, robotics, and other prediction tasks.

"These results demonstrate the importance of determining the hyperparameters of ESNs based on the time scale of the target time series and provide universal design guidelines for how to set the hyperparameters of ESNs," concludes Prof. Ikeguchi.

The authors also suggest that these findings require further validation to determine their applicability to various systems beyond chaotic dynamical systems.

Image title: Figure 1: Procedure for matching time scales of the target systems
Image caption: Researchers adjusted the time scales of different systems using decorrelation times to compare the echo state network (ESN) performance under similar temporal conditions.
Image link: https://doi.org/10.1587/nolta.17.998
Image credit: Professor Tohru Ikeguchi from Tokyo University of Science, Japan
License type: CC BY-NC-ND 4.0
Usage restrictions: Credit must be given to the creator. Only noncommercial uses of the work are permitted. Adaptations must be shared under the same terms.

Image title: Figure 2: Relationship between the target system's decorrelation time and the optimal spectral radius in an echo state network (ESN)
Image caption: The optimal spectral radius increased as the time scale of the target system became longer under both training conditions: (a) fixed training data points and (b) varying training data points, suggesting a guideline for selecting echo state network (ESN) settings.
Image link: https://doi.org/10.1587/nolta.17.998
Image credit: Professor Tohru Ikeguchi from Tokyo University of Science, Japan
License type: CC BY-NC-ND 4.0
Usage restrictions: Credit must be given to the creator. Only noncommercial uses of the work are permitted. Adaptations must be shared under the same terms.

About The Tokyo University of Science

Tokyo University of Science (TUS) is a well-known and respected university, and the largest science-specialized private research university in Japan, with four campuses in central Tokyo and its suburbs and in Hokkaido. Established in 1881, the university has continually contributed to Japan's development in science through inculcating the love for science in researchers, technicians, and educators.

With a mission of "Creating science and technology for the harmonious development of nature, human beings, and society," TUS has undertaken a wide range of research from basic to applied science. TUS has embraced a multidisciplinary approach to research and undertaken intensive study in some of today's most vital fields. TUS is a meritocracy where the best in science is recognized and nurtured. It is the only private university in Japan that has produced a Nobel Prize winner and the only private university in Asia to produce Nobel Prize winners within the natural sciences field.