Research Topics
Medical Data Mining Time Series Forecasting Computer Vision
Natural Language Processing Learning Analytics AI for Drug Design
Explainable AI (XAI) Optimization
Medical Data Mining
[JF006]. A novel multivariate time series forecasting dendritic neuron model for COVID-19 pandemic transmission tendency.
A novel coronavirus discovered in late 2019 (COVID-19) quickly spread into a global epidemic and, thankfully, was brought under control by 2022. Because of the virus’s unknown mutations and the vaccine’s waning potency, forecasting is still essential for resurgence prevention and medical resource management. Computational efficiency and long-term accuracy are two bottlenecks for national-level forecasting. This study develops a novel multivariate time series forecasting model, the densely connected highly flexible dendritic neuron model (DFDNM) to predict daily and weekly positive COVID-19 cases. DFDNM’s high flexibility mechanism improves its capacity to deal with nonlinear challenges. The dense introduction of shortcut connections alleviates the vanishing and exploding gradient problems, encourages feature reuse, and improves feature extraction. To deal with the rapidly growing parameters, an improved variation of the adaptive moment estimation (AdamW) algorithm is employed as the learning algorithm for the DFDNM because of its strong optimization ability. The experimental results and statistical analysis conducted across three Japanese prefectures confirm the efficacy and feasibility of the DFDNM while outperforming various state-of-the-art machine learning models. To the best of our knowledge, the proposed DFDNM is the first to restructure the dendritic neuron model’s neural architecture, demonstrating promising use in multivariate time series prediction. Because of its optimal performance, the DFDNM may serve as an important reference for national and regional government decision-makers aiming to optimize pandemic prevention and medical resource management. We also verify that DFDMN is efficiently applicable not only to COVID-19 transmission prediction, but also to more general multivariate prediction tasks. It leads us to believe that it might be applied as a promising prediction model in other fields.
[JF001]. A novel machine learning technique for computer-aided diagnosis.
The primary motivation of this paper is twofold: first, to employ a heuristic optimization algorithm to optimize the dendritic neuron model (DNM) and second, to design a tidy visual classifier for computer-aided diagnosis that can be easily implemented on a hardware system. Considering that the backpropagation (BP) algorithm is sensitive to the initial conditions and can easily fall into local minima, we propose an evolutionary dendritic neuron model (EDNM), which is optimized by the gbest-guided artificial bee colony (GABC) algorithm. The experiments are performed on the Liver Disorders Data Set, the Wisconsin Breast Cancer Data Set, the Haberman’s Survival Data Set, the Diabetic Retinopathy Debrecen Data Set and Hepatitis Data Set, and the effectiveness of our model was rigorously validated in terms of the classification accuracy, the sensitivity, the specificity, the F_measure, Cohen’s Kappa, the area under the receiver operating characteristic curve (AUC), convergence speed and the statistical analysis of the Wilcoxon signed-rank test. Moreover, after training, the EDNM can simplify its neural structure by removing redundant synapses and superfluous dendrites by the neuronal pruning mechanism. Finally, the simplified structural morphology of the EDNM can be replaced by a logic circuit (LC) without sacrificing accuracy. It is worth emphasizing that once implemented by an LC, the model has a significant advantage over other classifiers in terms of speed when handling big data. Consequently, our proposed model can serve as an efficient medical classifier with excellent performance.
[JC008]. DenseHashNet: a novel deep hashing for medical image retrieval.
With the wide application of imaging modalities such as X-ray and Computed Tomography (CT) in clini-cal practice, Content-based Medical Image Retrieval (CBMIR) has become a current research hotspot. Related studies have shown that hash-based image retrieval algorithms can retrieve relevant images faster and more accurately than traditional image retrieval methods. Therefore, in this paper, we propose a novel deep hashing method for medical image retrieval, called DenseHashNet. Specifically, we first use DenseNet to extract the original image features, and introduce the Spatial Pyramid Pooling (SPP) layer after the last Dense Block so that features at different scales can be extracted and multi-scale features fused with information from multiple regions. Then, the output of the SPP layer is subjected to Power-Mean Transformation (PMT) operation to enhance the nonlinearity of the model and improve the performance of the model. Finally, we map the output of PMT to hash codes through fully connected layers. Experimental results show that our method achieves better performance, compared with some representative methods.
Time Series Forecasting
[JF002]. Artificial immune system training algorithm for a dendritic neuron model. 
Dendritic neuron model (DNM), which is a single neuron model with a plastic structure, has been applied to resolve various complicated problems. However, its main learning algorithm, namely the back-propagation (BP) algorithm, suffers from several shortages, such as slow convergence rate, being easy to fall into local minimum and over-fitting problems. That largely limits the performances of the DNM. To address this issue, another bio-inspired learning paradigm, namely the artificial immune system (AIS) is employed to train the weights and thresholds of the DNM, which is termed AISDNM. These two methods have advantages on different issues. Due to the powerful global search capability of the AIS, it is considered to be efficient in improving the performance of the DNM. To evaluate the performance of AISDNM, eight classification datasets and eight prediction problems are adopted in our experiments. The experimental results and statistical analysis confirm that the AISDNM can exhibit superior performance in terms of accuracy and convergence speed when compared with the multilayer perceptron (MLP), decision tree (DT), the support vector machine with the linear kernel (SVM-l), the support vector machine with the radial basis function kernel (SVM-r), the support vector machine with the polynomial kernel (SVM-p) and the conventional DNM. It can be concluded that the reasonable combination of two different bio-inspired learning paradigms is efficient. Furthermore, for the classification problems, empirical evidence also validates the AISDNM can delete superfluous synapses and dendrites to simplify its neural structure, then transform the simplified structure into a logic circuit classifier (LCC) which is suitable for hardware implementation. The process does not sacrifice accuracy but significantly improves the classification speed. Based on these results, both the AISDNM and the LCC can be regarded as effective machine learning techniques to solve practical problems.
[JF006]
[JS007], [JS004], [JS003]
[JC014], [JC009], [JC007], [JC006], [JC004]
[CC004]
Computer Vision
[JF004]. A novel motion direction detection mechanism based on dendritic computation of direction-selective ganglion cells.
The visual system plays a vital role when the brain receives and processes information. Approximately ninety percent of the information received by the brain comes from the visual system, and motion detection is a crucial part of processing visual information. To further understand the generation of direction selectivity, we propose a novel apparent motion detection mechanism using direction-selective ganglion cells (DSGCs). Considering the simplicity of neural computation, each neuron is responsible for detection in a specific direction. For example, eight neurons are employed to detect movements in eight directions, and local information is collected by scanning. The global motion direction is obtained according to the degree of activation of the neurons. We report that this method not only has striking biological similarities with hypercomplex retinal cells, but can also make accurate discriminations. The pioneering mechanism may lead to a new technique for understanding more complex principles of the visual nervous system.
[JS002]. The mechanism of orientation detection based on color-orientation jointly selective cells.
This paper discusses the visual mechanism of global orientation detection and the realization of a mechanism-based artificial visual system for two-dimensional orientation detection tasks. For interpretation and practicability, we introduce the visual mechanism into the design of a detection system. We first propose an orientation detection mechanism according to the color-orientation jointly selectivity cortical neuron character. We assume that part of the orientation detection tasks is completed by the color-orientation jointly selective cells that are only responsible for orientation detection locally. Each cell can only be activated by stimuli with a specific orientation angle and the preferred color. We realize these cells by the McCulloch–Pitts neuron model and extend them to a two-dimensional version. In each local receptive field, there are four separate color-orientation jointly selective cells responsible for orientation detection, and their optimal responsive color corresponds to the central location’s color. Every local region connects such a set of cells. Subsequently, by these sets of these cells, we can collect all local information and obtain the global orientation according to the local activations. The type of local orientation angle recognized the most corresponds to the global orientation. Finally, a mechanism-based artificial visual system (AVS) is implemented. Several simulations and comparative experiments are provided to verify the effectiveness and generalization of the proposed orientation detection scheme and the superiority of the AVS to popular classification networks in orientation detection tasks. In addition, the feature extraction ability of AVS is shown to accelerate the learning and noise immunity of neural networks.
[JS005]
[JC016], [JC014], [JC012], [JC010], [JC008]
[CC019], [CC015], [CC013], [CC010], [CC008], [CC007], [CC006]
Natural Language Processing
[JC017]. MFLSCI: Multi-granularity fusion and label semantic correlation information for multi-label legal text classification.
Multi-label text classification tasks face challenges such as sample diversity, complexity, and the need for effective utilization of label correlations. In this paper, we propose a model that integrates multi-granularity fusion of text sequence features and label semantic correlation information. Our model leverages graph convolutional networks to extract label semantic correlation, which enhances classification performance for samples with similar labels and addresses label omission issues. Additionally, text convolutional neural networks are employed to extract multi-granularity sense group features from text sequences, calculate their similarity with semantic correlation label distributions, and dynamically adjust the similarity between text context and label information. This approach tackles the limitations of feature extraction in short texts and label confusion. We replace the original multi-hot label encoding in model training with a label distribution that fuses text multi-granularity sense group features and label correlation information, using a more precise encoding method for soft alignment based on label probability distributions. This enhances the model’s resilience to noisy data, avoiding the issue of assigning high-confidence probabilities to incorrect categories due to hard-coded supervision. Our model’s performance improvement on noisy datasets significantly surpasses that achieved by label smoothing. Extensive experiments on three legal text datasets and two generalized multi-label datasets demonstrate the model’s excellent performance. Our approach is applicable in various real-world scenarios, such as legal judgment prediction, news categorization, and recommendation systems, where accurate multi-label classification is crucial. Ablation and experiments on noisy datasets validate the model’s effectiveness and robustness.
[CC021], [CC020], [CC017], [CC016], [CC014], [CC011]
Learning Analytics
[JC013]. Attention-based artificial neural network for student performance prediction based on learning activities.
Student performance prediction was deployed to predict learning performance to identify at-risk students and provide interventions for them. However, prediction models should also consider external factors along with learning activities, such as course duration. Thus, we aim to distinguish the difference factor between the time dimension (duration of the course) and the feature dimension (students’ learning activities) by attention weights to provide helpful information and improve predictions of student performance. In this study, we introduce Attention-Based Artificial Neural Network (Attn-ANN), a novel model in educational data mining. The Attn-ANN combines attention weighting on the time and feature dimensions to examine the significance of lectures and learning activities and makes predictions by visualizing attention weight. We found that the Attn-ANN had a better area under the curve scores than conventional algorithms, and the attention mechanism allowed models to focus on input selectively. Incorporating the attention weighting of both the time and feature dimensions improved the prediction performance in an ablation study. Finally, we investigated and analyzed the model’s decision, finding that the Attn-ANN may be able to create synergy in real-world scenarios between the Attn-ANN’s predictions and instructors’ expertise, which underscores a novel contribution to engineering applications for interventions for at-risk students.
[CF003]. Visual analytics of learning behavior based on the dendritic neuron model.
Learning analytics, blending education theory, psychology, statistics, and computer science, utilizes data about learners and their environments to enhance education. Artificial Intelligence advances this field by personalizing learning and providing predictive insights. However, the opaque ’black box’ nature of AI decision-making poses challenges to trust and understanding within educational settings. This paper presents a novel visual analytics method to predict whether a student is at risk of failing a course. The proposed method is based on a dendritic neuron model (DNM), which not only performs excellently in prediction, but also provides an intuitive visual presentation of the importance of learning behaviors. It is worth emphasizing that the proposed DNM has a better performance than recurrent neural network (RNN), long short term memory network (LSTM), gated recurrent unit (GRU), bidirectional long short term memory network (BiLSTM) and bidirectional gated recurrent unit (BiGRU). The powerful prediction performance can assist instructors in identifying students at risk of failing and performing early interventions. The importance analysis of learning behaviors can guide students in the development of learning plans.
[CC021], [CC020], [CC017], [CC016], [CC012], [CC011]
AI for Drug Design
[JC015]. Improving the artificial bee colony algorithm with a proprietary estimation of distribution mechanism for protein-ligand docking.
The protein–ligand docking problem plays an essential role in structure-based drug design. The challenge for a protein–ligand docking method is how to execute an efficient conformational search to explore a well-designed scoring function. In this study, we improved the artificial bee colony (ABC) algorithm and proposed an approach called ABC-EDM to solve the protein–ligand docking problem. ABC-EDM employs the scoring function of the classical AutoDock Vina to evaluate a solution during docking simulation. ABC-EDM adopts the search framework of the canonical ABC algorithm to execute conformational search. By further investigating the characteristics of the protein–ligand docking problem, a proprietary search mechanism inspired by estimation of distribution algorithm, i.e., estimation of distribution mechanism (EDM), is designed to enhance the performance of ABC-EDM. To verify the effectiveness of the proposed ABC-EDM, we compare it with three variants of the ABC algorithm, three evolutionary computation algorithms, and AutoDock Vina. The experimental results show that ABC-EDM can effectively solve the protein–ligand docking problem, and it can achieve a success rate 5% higher than AutoDock Vina on the GOLD dataset. This study reveals that taking advantage of problem-specific information about the protein–ligand docking problem to enhance a docking method contributes to solving this problem.
Explainable AI (XAI)
[JS006]. A hardware-based orientation detection system using dendritic computation.
Studying how objects are positioned is vital for improving technologies like robots, cameras, and virtual reality. In our earlier papers, we introduced a bio-inspired artificial visual system for orientation detection, demonstrating its superiority over traditional systems with higher recognition rates, greater biological resemblance, and increased resistance to noise. In this paper, we propose a hardware-based orientation detection system (ODS). The ODS is implemented by a multiple dendritic neuron model (DNM), and a neuronal pruning scheme for the DNM is proposed. After performing the neuronal pruning, only the synapses in the direct and inverse connections states are retained. The former can be realized by a comparator, and the latter can be replaced by a combination of a comparator and a logic NOT gate. For the dendritic function, the connection of synapses on dendrites can be realized with logic AND gates. Then, the output of the neuron is equivalent to a logic OR gate. Compared with other machine learning methods, this logic circuit circumvents floating-point arithmetic and therefore requires very little computing resources to perform complex classification. Furthermore, the ODS can be designed based on experience, so no learning process is required. The superiority of ODS is verified by experiments on binary, grayscale, and color image datasets. The ability to process data rapidly owing to advantages such as parallel computation and simple hardware implementation allows the ODS to be desirable in the era of big data. It is worth mentioning that the experimental results are corroborated with anatomical, physiological, and neuroscientific studies, which may provide us with a new insight for understanding the complex functions in the human brain.
[CF001]. An evolutionary neuron model with dendritic computation for classification and prediction.
Advances in the understanding of dendrites promote the development of dendritic computation. For decades, the researchers are committed to proposing an appropriate neural model, which may feedback the research on neurons. This paper aims to employ an effective metaheuristic optimization algorithm as the learning algorithms to train the dendritic neuron model (DNM). The powerful ability of the backpropagation (BP) algorithm to train artificial neural networks led us to employ it as a learning algorithm for a conventional DNM, but this also inevitably causes the DNM to suffer from the drawbacks of the algorithm. Therefore, a metaheuristic optimization algorithm, named the firefly algorithm (FA) is adopted to train the DNM (FADNM). Experiments on twelve datasets involving classification and prediction are performed to evaluate the performance. The experimental results and corresponding statistical analysis show that the learning algorithm plays a decisive role in the performance of the DNM. It is worth emphasizing that the FADNM incorporates an invaluable neural pruning scheme to eliminate superfluous synapses and dendrites, simplifying its structure and forming a unique morphology. This simplified morphology can be implemented in hardware through logic circuits, which approximately has no effect on the accuracy of the original model. The hardwareization enables the FADNM to efficiently process high-speed data streams for large-scale data, which leads us to believe that it might be a promising technology to deal with big data.
[JF002]
[JS007], [JS003], [JS002]
[JC003], [JC002]
[CF003]
[CC018], [CC009], [CC004], [CC003]
Optimization
[JF005]. Dendritic neural network: a novel extension of dendritic neuron model.
The conventional dendritic neuron model (DNM) is a single-neuron model inspired by biological dendritic neurons that has been applied successfully in various fields. However, an increasing number of input features results in inefficient learning and gradient vanishing problems in the DNM. Thus, the DNM struggles to handle more complex tasks, including multiclass classification and multivariate time-series forecasting problems. In this study, we extended the conventional DNM to overcome these limitations. In the proposed dendritic neural network (DNN), the flexibility of both synapses and dendritic branches is considered and for-mulated, which can improve the model’s nonlinear capabilities on high-dimensional problems. Then, multiple output layers are stacked to accommodate the various loss functions of complex tasks, and a dropout mechanism is implemented to realize a better balance between the underfitting and overfitting problems, which enhances the network’s generalizability. The performance and computational efficiency of the proposed DNN compared to state-of-the-art machine learning algorithms were verified on 10 multiclass classification and 2 high-dimensional binary classification datasets. The experimental results demonstrate that the proposed DNN is a promising and practical neural network architecture.
[JF003]. A cuckoo search algorithm with scale-free population topology.
The scale-free network is an important type of complex network. The node degrees in a scale-free network follow the power-law distribution. In the skeleton of a scale-free network, there exists a few nodes which own huge neighborhood size and play an important role in information transmission of the entire network, while most of the network nodes have few connections whose influences of information exchange are limited to a relatively low level. In this paper, we introduce a scale-free population topology into the cuckoo search (CS) algorithm to propose a novel variant, which is termed the scale-free cuckoo search (SFCS) algorithm. Unlike other CS algorithms where the individuals exchange information randomly, two properties of the scale-free network can improve the SFCS algorithm in two aspects: the possibility that the information of competent individuals quickly floods the whole population is reduced significantly, which guarantees population diversity; and the corrupt individuals can learn from competent individuals with greater probability, which is beneficial for convergence. Thus, SFCS can obtain a better trade-off between exploitation and exploration during the search process. To evaluate the effectiveness of the proposed SFCS, 58 benchmark functions with different dimensions (10-D, 30-D, and 50-D), and 21 real-world optimization problems are employed in our experiment. We compare SFCS with the basic CS algorithm, two CS variants, and five state-of-the-art optimization algorithms, and the experimental results and statistical analysis verify the superiority of SFCS in terms of solution quality and convergence speed. Furthermore, we compare SFCS with a scale-free fully informed particle swarm optimization algorithm (SFIPSO) and the results prove our scale-free idea is effective despite its simplicity. We also introduce the scale-free population topology into the differential evolution (DE) and the firefly algorithm (FA) and the experimental results show that the scale-free population topology enhance the search ability of the DE and FA. These lead us to believe that the scale-free population topology may be a new technique for improving the performance of the population-based algorithms.
[JF002]
[JS001]
[JC009], [JC005], [JC003], [JC001]
[CF002]
[CC018], [CC005], [CC001]