Artificial intelligence for bone cancer imaging

Supervised versus unsupervised learning

There are two main learning styles in ML: supervised and unsupervised learning. Supervised learning involves datasets with known outputs. For instance, each entry within a dataset could have two features, such as tumor size and location, and a known output (called the label ), such as benign or malignant diagnosis. Supervised learning uses this dataset (called training data ) to train the model, generating a mapping function that connects input to output through pattern recognition, penalizing incorrect predictions, and correcting the model until a desired accuracy is achieved. In contrast, unsupervised learning involves datasets with no labels, where the computer extrapolates information and performs pattern recognition to generate outputs/predictions.

ML algorithms

Support vector machines (SVMs) are examples of supervised ML algorithms which create a decision boundary that separates two classes of data points in N-dimensional space (where N is the number of features in the training data). For instance, in a training dataset of 500 examples with each example having two features (e.g., tumor size and location), an SVM would consider a 2D space with 500 data points and segregate (e.g., benign and malignant tumors) by a decision boundary. A decision tree classifier is a classification algorithm that uses features to perform branching until it sufficiently segregates the training data. For instance, a branch point could be a tumor size >8 cm. This branch point creates a data subset which will be inputs to the next branch node. Random forest (RF) is another example of a classification algorithm, where individual decision trees become an ensemble. Each decision tree gives its prediction and the tree with the most votes becomes the prediction for the model. Naive Bayes classifier is a supervised classification algorithm based on Bayes' theorem that returns a probabilistic prediction given the inputs, assuming all features are independent.

Neural networks (NNs) are modeled after the architecture of the brain. Each neuron (or node) has incoming connections each associated with an assigned weight. When data come in through the connection, they are scaled by the corresponding weight. The node integrates the input values via an activation function, which determines whether to “fire” off the data to the next node. With this, input data are not evaluated independently but with the consideration of multiple different variables. Deep learning incorporates NNs with many more processing layers in the network.

CNNs are popular deep learning architectures for image recognition. This is because CNNs are effective at dimension reduction and images have high dimensionality because each pixel is considered a feature. CNNs use mathematical operations to process input data through multiple layers of interconnected networks, similar to the operations within the visual system of the human brain. When we look at an image, we look for certain characteristics to narrow down what the image can be. For a bone tumor image, we have a list of predefined features of what constitutes a malignant tumor, such as ill-defined borders, cortical destruction, aggressive periosteal reaction, etc. Recognizing these features, we classify the image as malignant bone tumor.

Neuroscientists David Hubel and Torsten Wiesel found that subpopulations of neurons in the visual cortex have different functions [ , ]: while some neurons detect edge orientation, other neurons determine different visual features. Together, these multiple layers of processing allow comprehensible image formation. To translate this to machine language, the machine uses pixels to delineate edges and curves within an image while CNNs process this information to outline specific structures, such as bones or tumors. CNNs provide image classifications with minimal data preprocessing and assign learnable weights to their multiple interconnected algorithms to define the importance of a specific feature, such as cortical destruction or bone matrix formation. Multiple layers in the algorithm can extract features from the input data such that the algorithm “learns” to predict a prespecified output, such as a certain bone tumor. When many convolutional layers are stacked together, they are called deep-CNNs.

Deep-CNNs use a variation of multilayer perceptrons (“neurons”) with shared-weights architecture and translation invariance characteristics to reconstruct high-level features from incomplete input data. Deep-CNNs can alleviate the need for explicit feature extraction, simplify and accelerate the feature selection process, and integrate the processes of feature extraction, selection, and supervised classification in the same deep architecture (called end-to-end learning ) [ ]. With these advantages, deep-CNNs are more accurate, faster, vendor independent, and less prone to “hallucination artifacts” (i.e., enhanced random signal effects) when compared to previous approaches. Even so, there are some limitations. Because CNNs are trained using overlapping patches of images across iterations, computational redundancy occurs and only local features in the image subsections can be extracted [ ]. Therefore, CNNs can be computationally expensive and miss global image semantics. Many enhancements, notably LeNet [ ], U-Net [ ], ResNet [ ], AlexNet [ ], and fully convolutional network (FCN) [ ], have been proposed to address some of CNNs' limitations. CNNs are continuously being refined and have shown promise in most, if not all, aspects of bone cancer imaging.