Improving the Classification Ability of Delegating Classifiers Using Different Supervised Machine Learning Algorithms

Cancer Classification & Prediction Is Vitally Important for Cancer Diagnosis. DNA Microarray Technology Has Provided Genetic Data That Has Facilitated Scientists Perform Cancer Research. Traditional Methods of Classification Have Certain Limitations E


Introduction
Cancer is a very severe disease described by tumors or out-of-control cell growth.The cancer facts and figures 2012-2015 showed that it is rapidly becoming the cause of thousands of deaths worldwide every year, A total of 1,658,370 new cancer cases were reported only in the United States in 2015 (Siegel et al., 2015).According to the World Health Organization (WHO) there were 8.2 million deaths from cancer throughout the world in 2012.Cancer classification is one of the major research areas in the medical field.Predicting precisely of different tumor types is a challenge and providing accurate prediction is worthwhile in providing better treatment to the patients (Rabindra kumar singh, 2015) To identity cancer causes and to discover the methods for prevention, identification, treatments and cure, the cancer research has prime importance and it is one of the major research areas in medical field (Mangasarian et al., 2009).A close examination of gene expression is important in many fields of biological research in order to retrieve the required statistics (Rabindra kumar singh, 2015).For a better insight into the problem posed by the cancer disease, real gene expression data of cancer patients is the key (Xue et al., 2004).DNA microarray technology has provided us with a lot of datasets available for various cancer types such as lymphoid cancer, colon cancer, and breast cancer (Lu and Han, 2003).In Machine Learning (ML) the reliability measure of a classifier is its accurate predictions on unseen data (Ferri et al., 2004).Decision Support Systems (DSSs) are knowledge-based systems, which refers to honor the domain knowledge so that it is responsive to automated reasoning, (Druzdzel and Flynn, 1999).Traditionally a DSS classifies or predicts by using a single classifier (Stefanowski, 2007).For complex problems the use of a single classifier results in the reduced accuracy and/or prediction rate of the overall classification task (Stefanowski, 2007).Among available strategies for improving the accuracy of classifiers, "Reject Option" classification using a single classifier is proposed in Machine Learning literature and all the examples within this rejection region are rejected by classifier (Nadeem et al., 2010).By increased rejection rate the prediction rate decreases in a single classifier"s classification (Stefanowski, 2007).In order to overcome the low prediction problem, it is proposed that the predictions of a committee are always better than the predictions of a single classifier (Ferri et al., 2004).To address the problems posed by reject-option in classification, the concept of Delegating classifiers has been proposed in the literature (Ferri et al., 2004).The idea behind the delegating classifiers is to send unclassified (Rejected) samples to the next more specialized and efficient classifier, Delegated Classifier (Ferri et al., 2004).The classifier which sends samples to the Delegated Classifier is referred to as Delegating Classifier.Various classifiers have been used in this research work that are proposed in Machine Learning Literature E.g.Support Vector Machines-SVMs (Vapnik, 1999), k-Nearest Neighbor-kNN (Fix and Hodges, 1951), Linear Discriminant Analysis-LDA (Fisher, 1936).Moreover, traditionally the same learner is used in delegation as "Delegating" and "Delegated" classifier e.g.Decision Tree learner (Ferri et al., 2004).The research work has investigated the effects of using different classifiers (SVM, LDA, kNN, etc.) as Delegating and Delegated classifiers for cancer prediction/classification. In this research, it is investigated to improve the classification.

Literature Review
Some samples are easier to to be classified than others and can be predicted using fewer variables as with a lower prediction cost (Blaise Hanczara, 2021).Machine learning models always make a prediction, even when it is likely to be inaccurate.This behavior should be avoided in many decision support applications, where mistakes can have severe consequences (Kilian Hendrickx, 23 July 2021).In classification with a reject option, the classifier is allowed in uncertain cases to abstain from prediction.(Vojtech Franc, 2023).According to Chow in order to reduce the error rate of a classifier, samples for which posteriori probabilities are insufficiently high should not be classified.Generally, such samples fall in some region in which samples from different classes overlap and an ambiguity arises.Because of inherences of doubts and din in data, the bugs are usually cannot be tolerated.Reject option was presented to reject the samples for extreme misrecognition; it transforms misclassification into rejection (Chow, 1957).It is important to make predictions only when they are reliable.Practically, this results in the addition of a "reject" option to the classifiers.If the confidence to predict the appropriate class of the sample is lowest then the samples are rejected (Nadeem et al. 2010).In safety critical applications of decision support systems, prediction errors may lead to a serious loss.In such cases, estimating when the model makes an error can be as important as its average performance.These two objectives are taken into account in classification with a reject option when the classifier is allowed in uncertain cases to abstain from prediction (Vojtech Franc, 2023).A thoughtful classifier would delegate the problematic or inaccurate samples to other more efficient classifiers, or possibly more specialized, classifiers.Delegation is mostly preferable for complex classification because it has several advantages over other collaborative or combining methods: Improved efficiency, because individual classifier learns from smaller examples than the previous one, improved clearness, since each classification derives from a single classifier, an outlook to simplify the overall multi-classifier scenario by introducing delegation (Ferri et al., 2004).
With respect to the idea of delegating classifiers, a step by step procedure is generally designed, in which a classifier decides which of the samples to be categorized and which to reject.The unclassifiable rejected samples are sent to other more efficient classifiers (Ferri et al., 2004).The next more efficient classifier in the chain gets trained on the delegated rejected examples and carries out further classification (Ferri et al., 2004).In the delegation classifier we get trained on training samples, predict the classes of new data with high confidence and delegate the samples with low confidence to the classifier 2 (Temanni et al., 2007).The classifier 2 performs the same as the classifier 1 by predicting the classes of new data samples with high confidence and delegates the samples with low confidence (Temanni et al., 2007).It is an admitted fact that the pragmatic performance of classifiers is dependent on the dataset availed.It is very hard to predict which classifier will deliver its best (Sotoca et al., 2005).Machine learning literature has introduced different types of classifiers and every classifier has its own mode of learning, Support Vector Machines-SVMs (Vapnik, 1996), k-nearest neighbor-kNN (Fix and Hodges, 1951), Linear Discriminant Analysis-LDA (Fisher, 1936).However, the classification accuracy of the classifier can be improved by introducing a reject option with the classifier (Chow, 1957).

Materials and Methods
A huge amount of real microarray gene expression datasets are available nowadays that can be used for cancer classification.For example, lymphoid cancer data set (Shipp et al., 2002), lymphoid cancer data set is the microarray gene expression dataset of cancer patients suffering from DLBCL -Diffuse large B -cell Lymphomas.(Shipp et al., 2002) have measured 7129 gene expression levels from 58 DLBCL patient samples.Among 58 Diffuse large B-cell Lymphomas, 32 are healed patients; however, 26 patients suffer from fatal or stubborn disease.Colon cancer dataset (Alon et al., 1999), the data set contains gene expression levels of 40 tumor and 22 normal colon tissues for 6500 human genes.Breast cancer dataset (Golub et al., 1999), the data consists of 47 patients suffering from acute lymphoblastic leukemia and 25 patients suffering from acute myeloid leukemia.Individuals in 72 patients got bone marrow sections gained at the time of therapeutic interventions.Acute Lymphoblastic in the two classes (Nina and Lipo, 2007).It is used to rank features for microarray data (Jä ger et al., 2002), and for mass spectrometry data (IIya, 2005).Methods for computing feature ranks are the Correlation coefficient, which evaluate the dependability of variables to the consequences.

Classification Techniques
The literal meaning of the classification is the act of distributing things into classes or categories of the same type.The classification technique implemented in the research work is supervised learning classification technique.Supervised learning is concerned with model training with a set of labeled examples, and to predict labels for unlabeled examples ensuring maximum accuracy.Supervised learning uses density estimation according to which there is an arrangement in the input space in a way that certain patterns occur more often than other patterns (Alpaydın, 2014).

Classifiers
Machine learning literature has proposed various types of classifiers for classification purpose, each classifier has its own mode of operation or classification, and moreover classifier"s performance varies from data to data (Sotoca et al., 2005).Support Vector Machines belong to the class of various kernel methods Linear, Polynomial, Radial, and Sigmoid and are progressing in the statistical learning theory.Like all kernel-based learning algorithms they are a general-purpose learning machine, for example Svm Linear and specific kernel functions, for others problems (Hofmann, 2006).Linear discriminant analysis is a statistical approach for supervised dimensionality reduction and classification.It computes an optimal conversion by minimizing the "within class" distance and maximizing the "between class" distance simultaneously, thus achieving maximum class discrimination (Ye, 2007).KNN is one of the most fundamental and simple classification methods and should be one of the first choices for a classification study when there is little or no prior knowledge about the distribution of the data (Fix and Hodges, 1951).

Rejection Techniques -Ambiguity Rejection
To reduce the error rate of a classifier, samples for which posteriori probabilities are too high should not be classified.Generally, such samples fall in some regions in which samples from different classes overlap and an ambiguity arises.Because of uncertainties and noise inherent in data, the errors are generally unavoidable (Nadeem et al., 2010).

Delegation
By increased rejection rate the prediction rate decreases in single classifier classification (Stefanowski, 2007).To address the problems posed by reject-option in classification, the concept of Delegating classifiers has been proposed in the Machine Learning Literature (Ferri et al., 2004).The idea behind the delegating classifiers is to send unclassified rejected samples to the next more specialized and efficient classifier, the Delegated Classifier (Ferri et al., 2004).The classifier which sends samples to Delegated Classifier is referred to as Delegating Classifier.Leukemia (ALL), ALL is a subset of arrays from a large acute lymphoblastic leukemia study.Acute lymphoblastic leukemia ascends from 2 categories of lymphocytes: one is T-cell and the other is B-cell.The details of the datasets used in this work are summarized in Table -1

Data Preprocessing
One of the useful means to recognize relevant features for dimensionality fall/reduction is feature selection.(Isabelle and Andre, 2003;Huan and Lei, 2005).Data needs preprocessing in order to overcome some problems like overfitting.Overfitting is the situation when the model is heavily complex, which means to say that the model has too many parameters, greater than the number of observations (Refaeilzadeh et al., 2009).K-fold cross-validation is used in this research work to overcome this problem.The dataset is splitted into k approximately alike subgroups.The kth subgroup is used as a test or validation set and the remaining k-1 subgroups are used as training set in k fold cross validation (Refaeilzadeh et al., 2008).Student t-test, estimates whether the means of two classes are statistically unlike by computing a ratio between the difference of two class means and the unevenness of  The choice of a classification method has its certain importance.The choice of the classification method depends upon how reliable and efficient it is.A single classifier however classifies the samples efficiently and accurately.
A classifier with a reject option has a reliable and improved accuracy rate (details discussed earlier).However, by increased sample rejection the prediction rate is decreased.The overall prediction rate is improved, if the rejected samples are delegated to the next more efficient classifier.So that prediction rate as well as accuracy rate of overall classification task is improved.Results showed that in delegation the overall classification workload is divided among the classifiers, and it is proved to be very efficient.A classifier classifies the example for which it is specialized and delegates the rejected examples to the next more efficient classifier which is more efficient to classify them.However, it is the requirement of successful delegation that the classifier that gets the rejected examples should be more efficient and more reliable to successfully classify the rejected example.The research work has investigated the use of different learning algorithms in both delegating and delegated classifiers which further increased the accuracy and decreased the error rate.This research investigated the fact that each learning algorithm has its own specific details for learning and complexity of a problem is based on data too.

Conclusion and Main Outcomes
In the present study we investigated the effects of using different machine learning algorithms in the scenario of delegating classifiers and we have improved the classification ability of a decision support system.The improved classification ability is of prime importance in almost every field of life.For example, in tumor classification and therapy, a decision support system with improved classification ability helped improve the quality of care and facilitated by providing more accurate, effective, and reliable diagnosis and treatments, and by avoiding errors due to misclassification.

Figure 1 .
Figure 1.Simulation plan used in current study is present

Table 1 .
. Summaries of Datasets Used In Presented Research Work