computer-aided assessment of catheters and tubes on

This manuscript has been accepted for publication in Radiology: Artificial Intelligence, which is published by theRadiological Society of North America ( ©2020 RSNA)

Computer-Aided Assessment of Catheters and Tubes on RadiographsHow Good is Artificial Intelligence for Assessment?

Xin Yia,∗, Scott J. Adamsa, Robert D. E. Hendersona, Paul Babyna

aDepartment of Medical Imaging, University of Saskatchewan, 103 Hospital Dr, Saskatoon, SK, S7N 0W8 Canada

Abstract

Catheters are the second most common abnormal finding on radiographs. The position of catheters must be as-sessed on all radiographs, as serious complications can arise if catheters are malpositioned. However, due to thelarge number of radiographs performed each day, there can be substantial delays between the time a radiographis performed and when it is interpreted by a radiologist. Computer-aided approaches hold the potential to assistin prioritizing radiographs with potentially malpositioned catheters for interpretation and automatically insert textindicating the placement of catheters in radiology reports, thereby improving radiologists’ efficiency. After 50years of research in computer-aided diagnosis, there is still a paucity of study in this area. With the development ofdeep learning approaches, the problem of catheter assessment is far more solvable. Therefore, we have performeda review of current algorithms and identified key challenges in building a reliable computer-aided diagnosis systemfor assessment of catheters on radiographs. This review may serve to further the development of machine learningapproaches for this important use case.

Keywords: Machine learning, Artificial intelligence, Radiographs, X-rays, Catheter, Tube, Computer-aideddetection

1. Introduction

A variety of intravascular catheters and gastrointestinal and airway tubes (all henceforth referred to as cathetersfor simplicity) are used in clinical practice for life-supporting purposes, especially in critically-ill patients in inten-sive care units (ICUs) and in emergency departments (1, 2, 3, 4, 5, 6). For example, endotracheal tubes (ETTs)are used to assist in lung ventilation and may prevent aspiration; umbilical venous catheters may be used for ad-ministration of fluids or medications in neonates (7). In order for catheters to be used safely, proper placementis essential as serious complications can arise when a catheter is malpositioned. Radiography is routinely used toassess catheter positioning after catheter insertion due to the wide availability and cost-effectiveness of this imag-ing modality. However, due to the large number of radiographs performed each day, there can be a substantialdelay in time between when a radiograph is taken and when it is interpreted by a radiologist. Table 1 presentscommonly used catheters, a brief description of each, and an estimate of the frequency with which they mightrequire repositioning following a confirmatory radiograph. A computer-aided detection (CAD) system that canautomatically detect and localize catheter placement offers several potential benefits to radiologists and radiologydepartments, including appropriate prioritization of cases with potentially malpositioned catheters in worklists forinterpretation, thereby shortening the turnaround time for critical cases, and automatically inserting text indicatingthe placement of catheters in radiology reports, thereby improving radiologists’ efficiency. Another possible appli-cation of computer-assisted approaches for catheter detection may be the processing of radiographs to remove allcatheters from an image in preparation for analysis by a CAD system for detecting pathology. This may be helpfulas current CAD systems for detection of pathology are often trained on images without any catheters present, and

Xin Yi and Scott J. Adams contributed equally to this work.∗Corresponding authorEmail addresses: [email protected] (Xin Yi), [email protected] (Scott J. Adams),

[email protected] (Robert D. E. Henderson), [email protected] (Paul Babyn)

1

arX

iv:2

002.

0341

3v1

[ee

ss.I

V]

9 F

eb 2

020

https://pubs.rsna.org/journal/ai


Purpose Appropriate position Some potentialmalpositioned locations

Some potential complicationsand insertion problems

Estimate of theproportion ofmalpositionedcases

Endotracheal tubes (ETT)

Ventilation, airwaymaintenance

5 cm above carina when the head is in neutral position or T3 or T4 if the carina is notvisualized (in adults); mid trachea, approximately halfway between the inferior margin ofthe clavicles and the carina (in children); 1.5 cm above the carina (in neonates) (6, 7, 14)

Bronchus, esophagus Trauma, infection, aspiration,altered oral developmenta

5–28% (1)

Tracheostomy tubes

Ventilation, airwaymaintenance, bypassobstruction

Tip should be one-half to two-thirds of the distance from the stoma to the carina (6) In or out, esophagus Bleeding, clogging, infection,leaks, granulation

10% (2)

Chest tubes

Pneumothorax, pleural/extrapleural fluidcollection or drainage

Depends on underlying condition; generally 4th6th intercostal space, mid-anterioraxillary line; side hole should be medial to the inner margin of the ribs (6, 15)

Variable. Potentialmalpositions include heartor great vessels

Bleeding, nerve damage 30% (3)

Nasogastric tubes (NGT)

Feeding, gastric access Stomach (pyloric antrum) (16) Esophagus, trachea, lung,coiling

Aspiration, apnea, obstruction,irritation, trauma, perforation,infection

≤ 15% (4)

Central venous catheters (including internal jugular, subclavian, and femoral catheters and peripherally-inserted central catheters)

Medicationadministration, TPN,fluids, dialysis,monitoring

SVC or IVC (17) Right atrium, proximal toSVC or IVC, portal vein,right or left atria orventricles

Infection, migration,thrombosis, phlebitis

2-7% (5, 18)

Umbilical venous cathetera(UVC)

Emergency or long-termvascular access ofpre-term infants,transfusions

Inferior right atrium or IVCright atrial junction (7) Heart, portal vein, ductusvenous, umbilical vein,coiling, lung

Misdirection, infection,perforation, thromboembolicevent anywhere in systemiccirculation

≤ 77% (19)

Umbilical arterial cathetera(UAC)

Blood gas measurement,arterial blood pressuremonitoring

Thoracic aorta at T6T10 (high position) or L3L5 (low position); high position currentlypreferred (7)

Heart, umbilical artery,external or internal iliacartery, coiling

As UVC, plus vasospasm 35% (20)

a Neonates only. Note. – IVC = inferior vena cava, SVC = superior vena cava, TPN = total parenteral nutrition, UVC = umbilical venous catheter.

Table 1: Selected catheters and tubes commonly encountered on chest radiographs (21, 16, 22, 23).

such structures may potentially be a source of confusion for deep learning algorithms (8). Although the first CADsystems for evaluation of radiographs was proposed in the 1960s by Lodwick et al. (9), it was not until 2007 thatthe first specialized system for catheter placement evaluation was published (10, 11). In the last 12 years there havebeen some efforts towards evaluating catheter placement on radiographs, but a general approach that is suitable forall types of catheters has yet to be presented. Early efforts in computer-aided catheter placement evaluation werelimited by the feature representations of the catheter appearance and oversimplified assumptions of catheter’s geo-metric shape and position. Many of the related works use low-level image processing operations such as templatematching (12) based region growing and Hough transform based line fitting (13), which is often found to be fragile.On the other hand, radiographic features can exhibit many different forms of variation as demonstrated in Figure 1,as well as extraneous structures such as ECG lines or random coiling outside of the body, which can further com-plicate evaluation. In addition, catheters can have low contrast to background structures, and window levels mustoften be adjusted to better visualize catheters. Recently, there has been a paradigm shift from traditional rule-basedapproaches to machine learning based, or more specifically deep learning approaches in medical imaging, resultingin drastic improvement of both diagnostic accuracy and interpretation time. In this article we review the progresstowards computer-assisted catheter placement evaluation algorithms and identify key issues and future opportu-nities in designing such systems. This review article will serve to further the development of machine learningapproaches for this important use case, highlight particular challenges and opportunities, and provide radiologistsand data scientists with an overview of how current CAD systems for catheter detection work and what they mayachieve in the future.

2



(a)

(b) (c)

Chest tube

UVCUAC

NGT

ETT

Coiled catheter

Figure 1: Three examples of radiographs showing the variety of lines and the complexity of catheter shape and appearance. (a) an adult chestradiograph with a right-sided chest tube. (b) a neonatal chest/abdomen radiograph with an endotracheal tube, nasogastric tube, umbilicalarterial catheter, and umbilical venous catheter. (c) a neonatal chest/abdomen radiograph with a coiled umbilical arterial catheter, anumbilical venous catheter, and an endotracheal tube. ETT = endotracheal tube, NGT = nasogastric tube, UAC = umbilical arterial catheter,UVC = umbilical venous catheter.

2. Review

To be clinically useful, systems to evaluate catheter placement should be able to answer the five questions shownin Figure 2. We begin with the most straightforward question of whether a catheter is present in the radiograph andthen gradually extract more information, including the location of the catheter tip and catheter course (sequenceof points as a representation of the catheter). With this information, we can start to identify the type of catheterbased on both its appearance and geometric features (the catheter’s course). Finally, it is critical to assess whetherthe catheter is in a satisfactory position. Note that there are currently no general algorithms that are able to answerall the questions mentioned above for all types of catheters in one single system. We provide a brief overview ofthe traditional pipeline of the catheter placement evaluation systems as shown in Figure 3 and mark on the top leftcorner of each relevant step the relevant question it tries to answer.

In the following sections of this paper, we categorize the related literature into five sections, each correspondingto the five questions that need to be answered for an automatic catheter placement evaluation system. At the endof each section, we describe the metrics used in the evaluation of these algorithms and identify potential futureresearch directions and challenges.

2.1. Q1: Is a catheter present?

Determining the presence of a catheter should be the initial step. It is reasonable to assume no prior knowledgeabout which type of catheters might be present. In machine learning, this is a typical supervised binary classificationproblem. A curated labeled dataset is needed for the algorithm to learn, with one class representing radiographswith catheters and the other class representing radiographs with no catheters. Lakhani trained a deep convolutionalneural network from end to end on 180 images to determine the presence/absence of ETT on chest radiographs (24).They achieved an area under the curve (AUC) of 0.99 which is sufficiently accurate given the reasonably easy natureof this task, although there might be a risk of overfitting on this relatively small dataset.

Evaluation metric: The AUCthe integral measure of the receiver operating characteristic (ROC) curveis acommonly used metric for binary classification problems where positive and negative classes are balanced (25).

3



Q1: Is a catheter present?

[Desired output] 0 or 1 for all the interested catheters

Q2: Where is the tip of the catheter?

[Desired output] The tip location (xi, yi) of each interested catheter Ci

Q3: What is the course of the catheter?

[Desired output] A sequence of points{(xi1, yi1), · · · , (xin, yin)} for each catheter Ci

Q4: Which type of catheter is it?

[Desired output] A discrete number representing each potential type of catheter

Q5: Is the catheter in a normal/satisfactory position?

[Desired output] 0 or 1 for each potential presented catheter

Figure 2: Research questions needed to be answered for a system to evaluate catheter placement.

When positive and negative classes are imbalanced, a precision-recall curvewhich plots the positive predictivevalue against the true positive ratemore accurately illustrates an algorithm’s performance. A related metric, the F1score, is calculated as the harmonic mean of precision and recall.

2.2. Q2: Where is the tip of the catheter?

Locating the catheter tip is critical for evaluation of catheter placement. Catheter tips should be placed inspecific anatomic regions to make sure they function correctly and to minimize the risk of complications. Forexample, the tip of an ETT should be placed in the trachea well above the carina to minimize the risk of selectivebronchial intubation resulting in collapse of the contralateral lung. Generally, two ways of localizing the tip of thecatheter are described in the literature. The first is by tracing from an initial seed point, either manually selected orautomatically detected, until a criterion is fulfilled (26, 27), e.g. a sudden intensity drop in the searching direction.The second is by using heuristics. For example, Lee et al. proposed a supervised deep learning based method tolocate the tip of a peripherally inserted central catheter (PICC) (28). They segmented out the lung regions andPICC and defined the tip as the lowest endpoint of the PICC inside the lung region. In addition to PICCs, theseapproaches may also be applied for locating the tips of other types of catheters, such as tunneled central venouscatheters and catheters connected to ports, as the distal aspects of these catheters have a similar appearance. Avariety of conditions may result in the failure of these approaches, including coiling of the catheter and abnormalpositioning of the catheter relative to anatomic landmarks, such as when the tip is superior to the lungs. A thirdpossible approach which holds the most promise for catheter tip detection is direct regression as commonly used ingeneral object detection using a deep learning approach. For catheter tip detection, regression targets would be thetip location rather than the four corners of the bounding box as used for general object detection (29).

Evaluation metrics: The most common metrics to evaluate the accuracy of tip localization are the meanabsolute distance (MAD) and the root mean square distance (RMSD) between the detected and the ground-truthtip location as marked by a radiologist (28). An alternative measure is the tip detection ratio as proposed by Kelleret al. (10). This is defined as the ratio of the detected catheter tips that are within a predefined distance (such as 2.5mm as used by Keller et al.) of the ground-truth location.

2.3. Q3: What is the course of the catheter?

Simply ensuring the correct placement of the catheter tip is not always sufficient for assessment, as a cathetermay loop on itself or take other aberrant paths. An example is shown in Figure 1 (c). In this case, knowing the

4



Start

Preprocessinge.g. contrast enhancement

pose normalization, etc

Catheter present?e.g. classification End

ROI detectione.g. active contour model or CNN

Catheter segmentatione.g. SRCNN

Instance separation Seed selectione.g. manual/automatic

Catheter tracinge.g. template matching

Catheter classificatione.g. SVM

Position classificationnormal vs. abnormal

Tip locatione.g. simple rule-based

End

Preprocessed input

Segmented ROI

Catheter course

yes

noQ1

Q2

Q3

Q3

Q4 Q5

Figure 3: Conventional catheter placement evaluation pipeline (drawn in black). Steps connected using magenta lines offer an alternativepipeline to answer all five research questions. Note that steps marked with a dashed line have no published research as of yet. The questionnumbers indicated in top left corner of each box correspond to the questions posed in Figure 2. ROI = region of interest, RCNN = recurrentconvolutional neural networks, CNN = convolutional neural network, SVM = support vector machine. Better viewed in color.

complete course of the catheter is helpful for assessment. In order to accomplish this task, a starting seed has tobe selected. This has been previously accomplished by either manual selection (10) or automatic selection withtemplate matching in the neck region of a radiograph (30). A sequence of points on the catheter can be generatedby a template matching-based region growing method. This method involves computing the cross-correlation ofthe image data in the neighborhood of the seed with a predefined catheter intensity profile and finding the next pointalong the direction that gives the best fit, iterating this process until a stopping criterion is met (10, 30, 31). Insteadof using template matching in the intensity domain, Sheng et al. used the Hough transform on the local window

5



of the edge image to determine which direction to trace (32). The downside of this operation is the computationcomplexity that could prevent it from being used in a realtime environment. Kao et al. and Chen et al. used aneven simpler method where the next point is determined on a row basis (26, 27). The x location of the next pointxnext is the one with the highest intensity values among [xcurrent1, xcurrent, xcurrent+1]. All of the literature thatinvolves catheter tracing assumes the target catheter forms no loops. This is a valid assumption for ETTs but not forother types of catheters. Therefore, Yi et al., Mercan and Celebi, and Lee et al. formulated this as a segmentationproblem and modelled the catheter as a collection of points with no particular order (28, 33, 34). All three worksadopted convolution neural networks (CNNs) trained on their especially curated dataset. Mercan and Celebi useda patch-based CNN to segment chest tube and used non-uniform rational basis spline (NURBS) curve fitting toconnect discontinuous catheter segments (34). Lee et al. used a fully convolution neural network (FCN) to segmentPICCs and used a probabilistic Hough transform to post-process the potential discontinuities (28). Yi et al. used ascale recurrent convolution neural network (RCNN) to segment ETTs, NGTs, UACs and UVCs (33). It treated allcatheters of interest as a single class and iteratively refined the segmentation results. Although promising resultswere achieved, they have only answered part of the question: where are the catheters? The other part of the questionis to differentiate each instance of catheter from the detected results. Deep learning approaches then may also beused to determine catheter coiling, which could be achieved by supervised training of a deep learning system oninstance segmentation maps.

Evaluation metrics: All of the studies that have computed catheter course in order to determine the tip locationdid not explicitly evaluate the accuracy of the catheter course with the exception of (10). In that study, the authorscomputed the tracking accuracy and precision based on the tracked catheter midline and the groundtruth cathetermidline annotated by radiologists.

The typical evaluation metrics for binary segmentation are the pixel-based ROC and precision-recall curve.Following catheter instance segmentation, a line fitting could then be performed to determine the catheter course.To evaluate the accuracy of the catheter course, we can borrow metrics used in segmentation that measure theboundary accuracy, such as mean absolute distance and Hausdorff distance.

Important aspects to consider when reporting evaluation metrics for catheter detection are the diameter of thecatheter, the pixel resolution of the image, and the contrast to noise ratio of the images (related to the type of X-rayequipment, the X-ray parameters employed, and the X-ray absorption of the materials from which the catheters aremade), as each can affect the performance of an automated catheter detection system.

2.4. Q4: Which type of catheter is it?

Ramakrishna et al. conducted preliminary studies to differentiate NGTs and ETTs. The authors assume NGTsand ETTs have different intensity profiles, and can therefore be identified via simple template matching (31).This approach does not work with UACs and UVCs on pediatric radiographs since both catheters have the sameappearance and only differ in their course relative to anatomic landmarks. Geometric features need to be taken intoconsideration along with appearance features for this differentiation. Another approach for differentiating NGTsand ETTs is from (32). They assume ETTs always lie between the two lungs. Kao et al. and Chen et al. extract twosimple features from the traced catheter and used a support vector machine (SVM) classifier to classify a catheteras an ETT or NGT (26, 27).

Evaluation metrics: Since classification of catheters on radiographs is essentially a multi-class classificationproblem, a confusion matrixa type of contingency tablecan be computed and per-class classification accuracy canbe determined. In addition, per-class AUC can be determined, assuming that determining the type of each catheteris a binary classification problem (one-vs-all strategy).

2.5. Q5: Is the catheter appropriately positioned?

Determining whether a catheter is appropriately positioned relative to anatomical landmarks is critical for thesafe use of catheters. Singh et al. used Inception V3, ResNet50, and DenseNet in a Tensorflow framework toclassify enteric feeding tube placements on chest and abdominal radiographs as critical (i.e. bronchial insertion)or non-critical (i.e. with the feeding tube in the esophagus, stomach, or duodenum; appropriately coursing beyondthe field of view; or absent from the radiograph). Inception V3 (pretrained with color images from ImageNet and

6



trained using radiographs labeled as critical or non-critical enteric tube placements) had the highest AUC overall,with an AUC of 0.87 (95 confidence interval 0.800.94) (35). No reports to our knowledge exist regarding assessingthe position of multiple types of catheters on a single radiograph.

Evaluation metrics: Similar to Q1: Is a catheter present?, determining whether a catheter is appropriatelypositioned is generally a binary classification problem, and the AUC can appropriately describe performance whenpositive and negative classes are balanced. Literature regarding computer-aided evaluation of catheter placementas described above is summarized in Table 2. As further algorithms are developed and studies describing theirperformance are published, a meta-analysis of performance measures may be warranted.

Catheter type Literature Performance Remarks

Adult

ETT

(11) Q1-TP 0.94 FPs/image 0 Rule-based, 107 cases (33 ETT, 54 FT, 22 NGT)(36) Q1-TP 0.92 FPs/image 0.5 Rule-based, 107 training cases (33 ETT, 54 FT, 22 NGT), 121

test cases(32) Q1-TP 0.94 FPs/image 0.6 Rule-based, 107 cases (33 ETT, 54 FT, 22 NGT)(31) Q1-TP 0.74 FPs/image 0.09 Rule-based, 25 cases (17 both, 2 ETT, 6 none)(30) Q1-TP 0.93 FPs/image 0.02 Rule-based, 64 cases (20 both, 5 NGT, 8 ETT, 31 none )(26) Q1-AUC 0.88 Rule-based, 87 cases (43 with and 44 without ETTs)(24) Q1-AUC 0.99 Patched CNN, NURBS curve fitting, 268 cases (21 with and

247 without catheters)

NGT

(11) Q1-TP 0.82 FPs/image 0 As above(32) Q1-TP 0.82 FPs/image 0.5 As above(31) Q1-TP 0.77 FPs/image 0.16 As above(30) Q1-TP 0.84 FPs/image 0.02 As above

FT (11) Q1-TP 0.82 FPs/image 0 As above(32) Q1-TP 0.82 FPs/image 0.4 As above(35) Q5: AUC 0.82 to 0.87 Deep learning, 5475 cases (174 radiographs with bronchial

insertions and 5301 non-critical radiographs), including 4745cases for training, 630 for validation, and 100 for testing

Chest tube (10)Q2-Tip detection rate: 0.75Q3-Mean tracking accuracy:0.85 Rule-based, 5 cases (containing 12 catheters), no differentia-

tion of different type of catheters(34) Pixel based TP: 0.59 FP:0.0 Patched CNN, NURBS curve fitting, 268 cases (21 with and

247 without catheters)

PICC (10)Q2-Tip detection rate: 0.75Q3-Mean tracking accuracy:0.85 No differentiation of different type of catheters

(28) Q2- MAD: 3.10±2.03 mm, RMSE:3.71 mm Deep learning, 600 cases (400 training, 50 validation, 150 test)

Central venous tube (10)Q2-Tip detection rate: 0.75Q3-Mean tracking accuracy:0.85 No differentiation of different type of catheters

Tracheostomy tube (10)Q2-Tip detection rate: 0.75Q3-Mean tracking accuracy:0.85 No differentiation of different type of catheters

Pediatric

ETT (27)Q1-AUC: 0.94Q2-MAD: 1.89±2.01 mm Rule-based, 1334 cases (528 with and 816 without catheters)

(37) Q3-Pixel based precision-recall, Fβ = 0.80 Deep learning, 2549 cases (2515 in training and 34 in testingdatasets); all catheters are treated as the same class

NGT (37) Q3-Pixel based precision-recall, Fβ = 0.80 All catheters are treated as the same class

FT (38) Not reported As above

UAC (37) Q3-Pixel based precision-recall, Fβ = 0.80 All catheters are treated as the same class

UVC (37) Q3-Pixel based precision-recall, Fβ = 0.80 All catheters are treated as the same class

Table 2: Summary of publications on computer-aided evaluation of catheter placement. Note.Performance is summarized based on thefive questions (Q1 to Q5) posed in Figure 2. TP = true positive, FP false positive, AUC = area under the curve, ETT = endotrachealtube, NGT = nasogastric tube, FT = feeding tube, PICC = peripherally inserted central catheter, UAC = umbilical arterial catheter, UVC =umbilical venous catheter, CNN = convolution neural network, MAD = mean absolute distance, RMSE = root mean square error, NURBS= non-uniform rational basis spline.

7



3. Current limitations and areas for further development

While a number of approaches have been used to assist with aspects of catheter assessment, including catheterdetection, classification, tracing, and assessment of tip location, there are no clear systematic approaches to solvingthe placement evaluation problem. Previous studies have investigated only a subset of the types of catheters whichare commonly found on radiographs while ignoring the others. The proposed algorithms may demonstrate accept-able performance on the dataset the respective authors curated, but it is difficult to conclude that the algorithm willshow sufficient performance at other institutions without further application to other large-scale datasets with awide variety of cases and catheter profiles. Among all the catheter types, algorithms have demonstrated by far thehighest detection accuracy for ETTs, likely because they consistently lie in the neck and upper chest and have highcontrast to the background. NGTs and FTs, on the other hand, are much more challenging to detect due to obscu-ration by the cardiac silhouette and abdominal structures and their more variable position. Whether performancecan be maintained when ETTs are combined with other catheters remains to be determined.

Although catheters are frequently present on radiographs (39), there has been remarkably little overall progressin the use of machine learning approaches to their evaluation. One of the impediments to further development isthe lack of large annotated datasets to establish a reliable reference standard, not only for catheters but also forreference landmark organs such as the lungs, heart, and vertebral bodies. However, there is already some collectiveeffort in making a large number of radiographs available for the research community to use. For example, the Chest-Xray14 dataset containing 112,120 radiographs with 14 image-level class labels referring to chest abnormalities wasreleased in 2017 by the National Institutes of Health (NIH) (40). These datasets contain radiographs with cathetersso they can be annotated in a way that is suitable for catheter placement evaluation research. Such datasets maybe annotated with “strong” or “weak” labels (41). Strong labels generally refer to pixel-level annotation, includingpixel-level annotation of the catheter and, in some cases, landmark organs used as a reference to assess catheterlocation. Weak labels refer to labels for the entire image, such as a label that an ETT is present or that an ETT isappropriately positioned or malpositioned.

Annotating images with strong labels require more human effort; however, it is anticipated that considerablyless training data is required when using strong labels. Annotating line structures using pixel-level annotations(strong labels) takes considerably more effort than other anatomical or pathological regions. Besides pixel maps,there are other ways of annotation that require less time and effort, such as drawing polygonal bounding boxes.There are some tools designed for this purpose, such as PolygonRNN++ (42). However, because catheters aregenerally not localized and can span the whole radiographic image, bounding boxes are not as useful as they are inother general computer vision problems. Research productivity could be significantly boosted with the developmentof dedicated easy-to-use tools for line structure annotation.

A related area for further development is real-time catheter segmentation and catheter tip detection duringimage-guided procedures such as interventional radiology procedures or voiding cystourethrography, for example.Real-time catheter segmentation or tip detection may aid radiologists in tracking catheters to ensure they are ad-vanced to the correct locations without inadvertently creating injury during catheter insertion. Real-time catheterdetection may also aid in coregistration between 2D and 3D imaging modalities for 3D real-time virtual naviga-tion systems, automated respiratory motion compensation, and automatic collimation to the region of the cathetertip (43). Real-time catheter segmentation during fluoroscopic procedures poses a number of challenges, includingthe low signal-to-noise of fluoroscopic images, relatively high framerate (e.g. 7-30 frames per second), and thefact that as the catheter is manipulated it may change considerably in location and shape between two consecutiveframes. However, early work has shown promising results. Ambrosini et al. adapted a U-net model to segmenta catheter and guidewire using the current image and the three previous images as inputs. They achieved tip de-tection with a median tip distance error of 0.9 mm. The median centerline distance error was 0.2 mm and 85% ofthe frames were within 1 mm of centerline distance error (44). Wu et al. used a cascaded detection-segmentationmodel consisting of detection and segmentation CNNs to segment catheter tips in fluoroscopic images obtainedduring percutaneous coronary intervention procedures. Their approach achieved tip precision of 0.532 pixels, F1score of 0.939, false tracking rate of 0.800%, and missing tracking rate of 9.900% (45). However, the runningspeed of 4-5 frames per second using two NVIDIA TITAN Xp 12G graphics processing units is a current limitation

8



to applying this approach more broadly in clinical practice. Using a conventional machine learning approach, Mil-letari et al. sought to automatically detect electrophysiology catheters in fluoroscopic sequences using Laplacianof Gaussian and difference of Gaussian-based filters, a Top-Hat filter to discard candidates that do not fulfill spatialand geometric constraint characteristics of an electrode, clustering, and scoring criteria using a greedy algorithm.Their approach achieved a tip detection rate of 99.3% and a mean distance of 0.5 mm from manually labeled groundtruth centroids. However, the approach relies on knowing a priori the number of catheters in the image (46).

Another challenge facing the interventional radiology community is perceived time-lag (latency) between theproduction of images and the true position of the catheter as it is manipulated. While increasing the framerate allowsfor a more accurate indication of real-time positioning, it comes at the cost of increased radiation exposure (47).Deep learning approaches for image denoising and catheter detection may allow users to use a higher framerateand lower dose rate, potentially reducing radiation dose overall while maintaining high image quality. In additionto catheter detection on conventional fluoroscopy, there is also opportunity to develop catheter detection systemsfor magnetic resonance imaging (particularly in the context of MRI-guided interventional procedures), and multi-projection X-ray systems such as biplane imaging and tomosynthesis. While limited work has been conductedregarding catheter detection on computed tomography (CT), research may benefit from borrowing approaches usedfor detecting tube-like structures such as the airways on CT imaging (48).

In summary, computer-aided assessment of catheters on radiographs will require integration of a variety of ap-proaches to determine whether a catheter is present, where the tip of the catheter is, what the course of the catheteris, which type of catheter it is, and, ultimately, whether the catheter is in a satisfactory position. While currentresearch is still in the early phases, we anticipate seeing additional work related to catheter detection in the comingyears due to the ubiquity of catheters on radiographs and the potential clinical benefits of an automatic catheterplacement evaluation system. Integration of an automatic catheter placement evaluation system into the radiologyworkflow may promote patient safety and assist radiologists in generating accurate reports in a shorter amount oftime. Given that there is no public dataset for chest/abdomen radiographs in neonatesmany of whom have a widevariety of catheters and tubesand to support efforts in building a generalized solution for catheter detection andattract the attention of a wider variety of researchers, we are releasing a locally collected neonatal chest/abdomenradiograph dataset which contains 100 images and catheter annotation maps with this paper 1 (further details re-garding the dataset are provided in Supplemental Information).

References

[1] Christopher J O’Connor, Hansen Mansy, Robert A Balk, Kenneth J Tuman, and Richard H Sandler. Identifi-cation of endotracheal tube malpositions using computerized analysis of breath sounds via electronic stetho-scopes. Anesthesia & Analgesia, 101(3):735–739, 2005.

[2] Ulrich Schmidt, Dean Hess, Jean Kwo, Susan Lagambina, Elise Gettings, Farah Khandwala, Luca M Bi-gatello, and Henry Thomas Stelfox. Tracheostomy tube malposition in patients admitted to a respiratoryacute care unit following prolonged ventilation. Chest, 134(2):288–294, 2008.

[3] Francis Remerand, Virginie Luce, Yasmina Badachi, Qin Lu, Belaıd Bouhemad, and Jean-Jacques Rouby.Incidence of chest tube malposition in the critically illa prospective computed tomography study. Anesthesi-ology: The Journal of the American Society of Anesthesiologists, 106(6):1112–1119, 2007.

[4] Bruce W Thomas and Robert E Falcone. Confirmation of nasogastric tube placement by colorimetric indicatordetection of carbon dioxide: a preliminary report. Journal of the American College of Nutrition, 17(2):195–197, 1998.

[5] M Muhm, G Sunder-Plassmann, R Apsner, T Pernerstorfer, A Rajek, A Lassnigg, R Prokesch, K Derfler, andW Druml. Malposition of central venous catheters. incidence, management and preventive practices. WienerKlinische Wochenschrift, 109(11):400–405, 1997.

1https://github.com/xinario/PediatricXray100

9



[6] Myrna CB Godoy, Barry S Leitman, Patricia M de Groot, Ioannis Vlahos, and David P Naidich. Chest radiog-raphy in the icu: part 1, evaluation of airway, enteric, and pleural tubes. American Journal of Roentgenology,198(3):563–571, 2012.

[7] Nathan David P Concepcion, Bernard F Laya, and Edward Y Lee. Current updates in catheters, tubes anddrains in the pediatric chest: A practical evaluation approach. European journal of radiology, 95:409–417,2017.

[8] Ramandeep Singh, Mannudeep K Kalra, Chayanin Nitiwarangkul, John A Patti, Fatemeh Homayounieh,Atul Padole, Pooja Rao, Preetham Putha, Victorine V Muse, Amita Sharma, et al. Deep learning in chestradiography: detection of findings and presence of change. PloS one, 13(10), 2018.

[9] Gwilym S Lodwick, Theodore E Keats, and John P Dorst. The coding of roentgen images for computeranalysis as applied to lung cancer. Radiology, 81(2):185–200, 1963.

[10] Brad M Keller, Anthony P Reeves, Matthew D Cham, Claudia I Henschke, and David F Yankelevitz.Semi-automated location identification of catheters in digital chest radiographs. In Medical Imaging 2007:Computer-Aided Diagnosis, volume 6514, page 65141O. International Society for Optics and Photonics,2007.

[11] Zhimin Huo, Sheng Chen, D Foos, and Yuming Rao. Computer-aided detection of tubes and lines in portablechest x-ray images. International Journal of Computer Assisted Radiology and Surgery (Print), 2(Suppl.1):S370–S372, 2007.

[12] Roberto Brunelli. Template matching techniques in computer vision: theory and practice. John Wiley &Sons, 2009.

[13] Richard O Duda and Peter E Hart. Use of the hough transformation to detect lines and curves in pictures.Technical report, SRI INTERNATIONAL MENLO PARK CA ARTIFICIAL INTELLIGENCE CENTER,1971.

[14] Lawrence R Goodman, Peter A Conrardy, Faye Laing, and MORLEY M Singer. Radiographic evaluation ofendotracheal tube position. American Journal of Roentgenology, 127(3):433–434, 1976.

[15] Stefan Huber-Wagner, Markus Korner, Achim Ehrt, Mike V Kay, Klaus-Jurgen Pfeifer, Wolf Mutschler,and Karl-Georg Kanz. Emergency chest tube placement in trauma care—which approach is preferable?Resuscitation, 72(2):226–233, 2007.

[16] Sanjay N Jain. A pictorial essay: Radiology of lines and tubes in the intensive care unit. The Indian journalof radiology & imaging, 21(3):182, 2011.

[17] M Nayeemuddin, AD Pherwani, and JR Asquith. Imaging and management of complications of centralvenous catheters. Clinical Radiology, 68(5):529–544, 2013.

[18] Wolfram Schummer, Claudia Schummer, Norman Rose, Wolf-Dirk Niesen, and Samir G Sakka. Mechanicalcomplications and malpositions of central venous cannulations by experienced operators. Intensive caremedicine, 33(6):1055–1059, 2007.

[19] Anne Ades, Craig Sable, Susan Cummings, Russell Cross, Bruce Markle, and Gerard Martin. Echocardio-graphic evaluation of umbilical venous catheter placement. Journal of perinatology, 23(1):24, 2003.

[20] David A Oppenheimer, Barbara A Carroll, Karen E Garth, and Bruce R Parker. Sonographic localization ofneonatal umbilical catheters. American Journal of Roentgenology, 138(6):1025–1032, 1982.

[21] Mhairi G MacDonald, Jayashree Ramasethu, and Khodayar Rais-Bahrami. Atlas of procedures in neonatol-ogy. Lippincott Williams & Wilkins, 2012.

10



[22] C Green and MD Yohannan. Umbilical arterial and venous catheters: placement, use, and complications.Neonatal network: NN, 17(6):23–28, 1998.

[23] Jayashree Ramasethu. Complications of vascular catheters in the neonatal intensive care unit. Clinics inperinatology, 35(1):199–222, 2008.

[24] Paras Lakhani. Deep convolutional neural networks for endotracheal tube position and x-ray image classifi-cation: Challenges and opportunities. Journal of digital imaging, 30(4):460–468, 2017.

[25] Tom Fawcett. An introduction to roc analysis. Pattern recognition letters, 27(8):861–874, 2006.

[26] Sheng Chen, Min Zhang, Liping Yao, and Wentao Xu. Endotracheal tubes positioning detection in adultportable chest radiography for intensive care unit. International journal of computer assisted radiology andsurgery, 11(11):2049–2057, 2016.

[27] E-Fong Kao, Twei-Shiun Jaw, Chun-Wei Li, Ming-Chung Chou, and Gin-Chung Liu. Automated detectionof endotracheal tubes in paediatric chest radiographs. Computer methods and programs in biomedicine,118(1):1–10, 2015.

[28] Hyunkwang Lee, Mohammad Mansouri, Shahein Tajmir, Michael H Lev, and Synho Do. A deep-learning sys-tem for fully-automated peripherally inserted central catheter (picc) tip detection. Journal of digital imaging,pages 1–10, 2017.

[29] Zhong-Qiu Zhao, Peng Zheng, Shou-tao Xu, and Xindong Wu. Object detection with deep learning: A review.IEEE transactions on neural networks and learning systems, 30(11):3212–3232, 2019.

[30] Bharath Ramakrishna, Matthew Brown, Jonathan Goldin, Christopher Cagnon, and Dieter Enzmann. Animproved automatic computer aided tube detection and labeling system on chest radiographs. In MedicalImaging 2012: Computer-Aided Diagnosis, volume 8315, page 83150R. International Society for Optics andPhotonics, 2012.

[31] Bharath Ramakrishna, Matthew Brown, Jonathan Goldin, Chris Cagnon, and Dieter Enzmann. Catheterdetection and classification on chest radiographs: an automated prototype computer-aided detection (cad)system for radiologists. In Medical Imaging 2011: Computer-Aided Diagnosis, volume 7963, page 796333.International Society for Optics and Photonics, 2011.

[32] Chen Sheng, Li Li, and Wang Pei. Automatic detection of supporting device positioning in intensive care unitradiography. The International Journal of Medical Robotics and Computer Assisted Surgery, 5(3):332–340,2009.

[33] X Yi, Scott Adams, Paul Babyn, and Abdul Elnajmi. Automatic catheter and tube detection in pediatric x-rayimages using a scale-recurrent network and synthetic data. Journal of digital imaging, pages 1–10, 2019.

[34] Cem Ahmet Mercan and Mustafa Serdar Celebi. An approach for chest tube detection in chest radiographs.IET Image Processing, 8(2):122–129, 2014.

[35] Varun Singh, Varun Danda, Richard Gorniak, Adam Flanders, and Paras Lakhani. Assessment of criticalfeeding tube malpositions on radiographs using deep learning. Journal of digital imaging, 32(4):651–655,2019.

[36] Zhimin Huo, Simon Li, Minjie Chen, and John Wandtke. Computer-aided interpretation of icu portable chestimages: automated detection of endotracheal tubes. In Medical Imaging 2008: Computer-Aided Diagnosis,volume 6915, page 69152J. International Society for Optics and Photonics, 2008.

[37] Xin Yi, Scott Adams, Paul Babyn, and Abdul Elnajmi. Automatic catheter detection in pediatric x-ray imagesusing a scale-recurrent network and synthetic data. Journal of digital imaging, 2018.

11



[38] Kendall Lee Bingham. Pseudo random forests for tube identification. Master’s thesis, 2015.

[39] Bram van Ginneken, Laurens Hogeweg, and Mathias Prokop. Computer-aided diagnosis in chest radiography:Beyond nodules. European Journal of Radiology, 72(2):226–230, 2009.

[40] Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localiza-tion of common thorax diseases. In 2017 IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pages 3462–3471. IEEE, 2017.

[41] Eirini Arvaniti and Manfred Claassen. Coupling weak and strong supervision for classification of prostatecancer histopathology images. arXiv preprint arXiv:1811.07013, 2018.

[42] David Acuna, Huan Ling, Amlan Kar, and Sanja Fidler. Efficient interactive annotation of segmentationdatasets with polygon-rnn++. arXiv preprint arXiv:1803.09693, 2018.

[43] YingLiang Ma, Mazen Alhrishy, Srinivas Ananth Narayan, Peter Mountney, and Kawal S Rhode. A novelreal-time computational framework for detecting catheters and rigid guidewires in cardiac catheterizationprocedures. Medical physics, 45(11):5066–5079, 2018.

[44] Pierre Ambrosini, Daniel Ruijters, Wiro J Niessen, Adriaan Moelker, and Theo van Walsum. Fully automaticand real-time catheter segmentation in x-ray fluoroscopy. In International Conference on Medical ImageComputing and Computer-Assisted Intervention, pages 577–585. Springer, 2017.

[45] Yu-Dong Wu, Xiao-Liang Xie, Gui-Bin Bian, Zeng-Guang Hou, Xiao-Ran Cheng, Sheng Chen, Shi-Qi Liu,and Qiao-Li Wang. Automatic guidewire tip segmentation in 2d x-ray fluoroscopy using convolution neuralnetworks. In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1–7. IEEE, 2018.

[46] Fausto Milletari, Nassir Navab, and Pascal Fallavollita. Automatic detection of multiple and overlappingep catheters in fluoroscopic sequences. In International Conference on Medical Image Computing andComputer-Assisted Intervention, pages 371–379. Springer, 2013.

[47] Stephen Balter. Fluoroscopic frame rates: not only dose. American Journal of Roentgenology, 203(3):W234–W236, 2014.

[48] Qier Meng, Takayuki Kitasaka, Yukitaka Nimura, Masahiro Oda, Junji Ueno, and Kensaku Mori. Automaticsegmentation of airway tree based on local intensity filter and machine learning technique in 3d chest ctvolume. International journal of computer assisted radiology and surgery, 12(2):245–261, 2017.

12


computer-aided assessment of catheters and tubes on

Documents