How can we address the above challenges through AI/ML?

Right now, Machine Learning and in particular Deep Learning (DL) has begun having tremendous effects across various areas in healthcare. The rising availability of medical information and fast improvement in techniques have made it possible to have these outcomes [4]. DL techniques can reveal clinically relevant information hidden in large swathes of health care data, which in turn can be used for decision making, treatment, control, and prevention of health conditions [4]. Some application areas of DL include health behavior reaction, EHR processing and retrieving scientifically sound treatment from text, eye related analysis and classification, cancer treatment, heart diagnosis [and brain activity analysis [4].This makes the treatment simpler for doctors and advantageous for patients, with quicker and productive monitoring.

Let us look at how Machine Learning and Deep Learning can solve some of the challenges mentioned in the previous section, and deep dive into the details of two specific cases: Skin Cancer Classification and role of AI in COVID-19.

1. Personalised Medicine : Advances in ML have made it possible to learn individualized treatment effects from observational data,i.e, data collected from clinical practice. Generally, such learning is a really challenging problem since the data is usually biased (clinicians choose which patients should receive the drug/treatmentrather than randomising), counterfactuals (what the outcome would have been if a treated patient had not been treated or conversely if a non-treated patient had been treated) are not observed, and both the decisions and the outcomes may be affected by hidden confounders (not recorded in the observational data) [3]. Also, since counterfactuals are not observed, predictive models cannot be tested out of sample. However, recent work in ML has made some inroads in this problem area. Stefan Wager and Susan Athey have adapted Random Forest method where predictions come with statistically valid confidence intervals [5]. Ahmed Alaa and Mihaela van der Schaar have developed general guidelines for creating ML algorithms that estimate individualised treatment effects by characterising the basic limits of what can be achieved and establishing conditions under which these fundamental limits can be realized [6]. Their analysis shows that the importance of different features of observational data vary with the sample size. In the case of small sample sizes, selection bias is the most important bottleneck, whereas in large sample sizes, modelingof the control and treated outcomes is the biggest bottleneck. Building on these findings — and others that result from the theoretical analysis [6,7]– they develop practical ML-algorithms that outperform previous methods.

2. Risk Scoring and Prognosis : As mentioned in previous section, risk scoring is not systematic enough, or even if it is, it usually follows linear models. ML methods are able to better handle these tasks due to two reasons. First is an information gain: that ML-based methods can handle a large number of features. This is important because the more the number of features that can be analysed, the better risk prediction becomes as different features contribute differently to risk for different types of patients. The second reason is a modelling gain: ML-based methods are able to make better use of the same features by better identifying and capturing the complex interactions between these features [3]. These gains allow ML-based methods to generate more accurate predictions, and hence better treatment advice for each patient. This shows why ML-based models can outperform linear models in diseases where there are large number of variables involved, with highly complex underlying interactions between them. Another advantage of ML-based models is that they can discover importance of features and /or interactions among them that were not previously understood, or thought to be important [3]. For example, the work of Alaa discovered an unexpectedly important role for oxygenation — in addition to FEV1 (forced expiratory volume) — in predicting the decline of patients suffering from Cystic Fibrosis [8].

3. Monitoring and Early Warning Systems : ML methods have made real progress in this are. Saria et al have developed ML methods that provide predictions of septic shock that are more accurate and more timely than those provided by commonly-used clinical models [9]. Van der Schaar et al have developed ML methods that provide predictions of cardiac arrest in hospital that are more accurate and more timely than those determined by existing clinical models [10].The techniques used in these two scenarios are very different, largely because they present very different kinds of conditions. But there are many kinds of sudden deterioration in hospital, and creating a predictive model for each one would present very difficult task. That remains a challenge for developing more such systems.

Now let us take a look into two case studies to see how methods in AI have helped in the respective scenarios.

Case Study 1: Skin Cancer Classification with Deep Neural Networks

Esteva [11] have developed a dermatologist-level skin cancer classification system using Convolution Neural Networks (CNN). A CNN consists of input layer, multiple hidden layers and output layers of neurons or units. Each connection between units across layers carries a weight. The weights are updated by minimizing an optimization function that tries to reduce the difference between original output and the predicted output of neural network.

CNN (Source: Tobore 2019 )

In their work, the authors used open access datasets which consisted of images of skin lesions. These images were labeled as either malignant or benign. They used images of lesions from different view points, and care was taken to ensure there was no overlap of images of same lesion from multiple points of view existed in test and training set. The dataset consisted of 129450 skin lesions comprising 2032 different diseases, which were mapped to 757 training clases based on medical taxonomy of skin diseases [11].

For training, they used a modified version of Google’s Inception v3 CNN architecture [12]. They removed the final classification layer from the network and retrained it with their dataset, fine-tuning the parameters across all layers. During training they resized each image to 299 × 299 pixels in order to make it compatible with the original dimensions of the Inception v3 network architecture and to leverage the natural-image features learned by the ImageNet pre-trained network. This procedure is known as transfer learning [11].

Source : Esteva [11]

The CNN was trained using backpropagation. All layers of the network were fine-tuned using the same global learning rate of 0.001 and a decay factor of 16 every 30 epochs. RMSProp [13] was used as the optimisation algorithm with a decay of 0.9, momentum of 0.9 and epsilon of 0.1. Google’s TensorFlow [14] deep learning framework was used to train, validate and test the network. During training, images were augmented by a factor of 720. Each image was rotated randomly between 0° and 359°. The largest upright inscribed rectangle is then cropped from the image, and it was flipped vertically with a probability of 0.5 [11].

To take advantage of fine-grained information contained within the taxonomy structure, the algorithm is trained to partition diseases into fine-grained training classes (for example, amelanotic melanoma and acrolentiginous melanoma). During inference, the CNN outputs a probability distribution over these fine classes. To recover the probabilities for coarser-level classes of interest (for example, melanoma) we sum the probabilities of their descendants, for example by summing over amelanotic melanoma and acrolentiginous melanoma [11].

The effectiveness of the algorithm was validated using 9 fold cross validation. First, the algorithm was validated using a three-class disease partition — the first-level nodes of the taxonomy, which represent benign lesions, malignant lesions and non-neoplastic lesions [11]. In this task, the CNN achieves 72.1 ± 0.9% (mean ± s.d.) overall accuracy and two dermatologists attain 65.56% and 66.0% accuracy on a subset of the validation set [11]. Second, they validated the algorithm using a nine-class disease partition — the second-level nodes — so that the diseases of each class have similar medical treatment plans. The CNN achieves 55.4 ± 1.7% overall accuracy whereas the same two dermatologists attain 53.3% and 55.0% accuracy [11].

Finally the direct performance of the CNN was tested against a board of 21 certified dermatologists. The metric chosen to compare was sensitivity and specificity, where sensitivity is ratio of true positives over all positives and specificity is ratio of true negatives over all negatives. The CNN achieves performance comparable with all tested experts, demonstrating an artificial intelligence capable of classifying skin cancer with a level of competence comparable to dermatologists [11].

Case Study 2: COVID and AI

The world is reeling under a global pandemic, the likes of that has never been seen before. The novel Coronavirus disease has spread across countries, affecting millions of people directly, and every single one of us indirectly.

In this scenario, Artificial Intelligence has actually played a huge role in containing the disease wherever and however it has been possible. Even before the threat was known to the world, AI systems had detected the outbreak of an unknown pneumonia type virus. As the outbreak has now become a global pandemic, AI tools and technologies have been employed to support efforts of policy makers, the medical community, and society at large to manage every stage of the crisis and its aftermath: detection, prevention, response, recovery and to accelerate research.

Source: OECD

Accelerating research using AI to understand and treat COVID-19

· Various methods in AI have helped governments and the medical community understand the COVID-19 virus and propel research on treatments by rapidly analysing massive amounts of research material. Natural Language Processing (NLP) algorithms, which can analyse large amounts of textual data have uncovered the virus’ history, transmission, and diagnostics, management measures, and lessons from previous epidemics.

· Deep learning models help in predicting several kinds of drugs or treatment methods that might treat COVID-19. DeepMind and several other organisations have used deep learning to predict the structure of proteins associated with SARS-CoV-2, the virus that causes COVID-19 [15].

· Platforms dedicated to this cause allows the consolidation and sharing of multidisciplinary expertise on AI, including on a global scale. The US government for instance started an exchange with worldwide governments and innovators that incorporates utilizing AI to quicken investigation of coronavirus research material made accessible using the Kaggle website [15]. Access to datasets in epidemiology, bioinformatics and related fields is being released, e.g. through the COVID-19 Open Research Dataset Challenge by the US government and partner organisations that makes available over 29 000 academic articles for coronavirus and COVID-19 [15].

· Computing power for to run ML/DL algorithms for COVID forecast and research is also being made accessible to everyone by big technology companies such as Amazon, Microsoft and Google, by public-private efforts like the COVID-19 High Performance Computing Consortium and AI for Health private and private individuals donating computing power (e.g. Folding@home) [15].

· Innovative approaches such as hackathons, prizes and open-source collaborations are helping to accelerate research on AI-based solutions for the pandemic. For example, the United Kingdom’s “CoronaHack — AI vs. Covid-19” seeks ideas from data scientists, businesses and biomedical researchers on using AI to control and manage the pandemic [15].

Using AI to detect, diagnose and prevent the spread of the coronavirus

AI can also be utilised to help recognise, diagnose and prevent the reach of the pandemic. ML/DL algorithms that identify patterns and anomalies are already working to detect and predict the spread of COVID-19, while image recognition systems are speeding up medical diagnosis. For example:

· AI based early warning systems have been able to understand and detect epidemiological patterns by mining media and online content and other information sources in several languages. Eg: WHO Early Warning System.

· AI tools can help identify virus transmission chains and monitor broader economic impacts [. In many scenarios, AI based technologies have demonstrated their ability to understand epidemiological data more rapidly than traditional reporting of health data. Institutions such as Johns Hopkins University and the OECD ( have also made available interactive dashboards that track the virus’ spread through live news and real-time data on confirmed coronavirus cases, recoveries, and deaths [15].

· Rapid diagnosis is critical to restrict infections and understand its spread. Applied to symptom data and images, AI has helped to rapidly diagnose COVID-19 cases [15]. Attention is being given to collecting data that is representative of the entire population to guarantee scalability and precision.

· Various nations are utilising population surveillance to monitor COVID-19 cases (for example, in South Korea algorithms use camera footage, geolocation data and credit card records to track and trace coronavirus patients). China assigns a risk level (colour code — red, yellow or green) to each person having infection risk using cell phone software [15]. While machine learning models use travel, payment, and communications data to predict the location of the next outbreak, and inform border checks, search engines and social media are also helping to track the disease in real-time [15].

· Several countries, such as Austria, China, Israel, Poland, Singapore and South Korea have set up contact tracing systems to detect possible contagion routes. In Israel, for example, geolocation data was used to identify people coming into close contact with known virus carriers, and send them text messages directing them to isolate themselves immediately [15].

· Semi-autonomous robots and drones are being deployed to respond to immediate needs in hospitals such as delivering food and medications, cleaning and sterilisation, aiding doctors and nurses, and performing deliveries of equipment.