Early diagnosis and treatment of malocclusion in mixed dentition is one of the important topics in pediatric dentistry. Lateral cephalometric radiography is an essential element in evaluating the cause of malocclusion by determining the relationship between the teeth, skeleton and soft tissue[1]. The pediatric dentist’s judgment is important in establishing a treatment plan and recognizing patients’ needs. These orthodontic treatment decisions may vary between pediatric dentists due to differences of opinion. Clinicians with less experience have trouble making the right decision[2].

Manual tracing is time-consuming and comes with a high risk of misreading values. Manual tracing is susceptible to error in landmark identification due to radiographic film magnification[3,4]. Manual tracing has been replaced by digital tracing gradually. Computer-assisted orthodontic analysis, such as VCeph (Osstem, Seoul, Korea), FACAD (Ilexis AB, Linköping, Sweden), and Dolphin (Dolphin Imaging, Chatsworth, CA, USA), has many advantages: the ease of use, unnecessariness of hard copies, and modification of digital images[5]. In spite of these advantages, several factors influence the selection of software, such as the complexity of installation procedures, high subscription and update cost, and the significant effort required to master the training software[6].

According to recent studies, the clinical adaptation of deep learning models has shown outstanding performance in dentistry[7,8]. Many algorithms have been developed to automatically detect cephalometric landmarks for orthodontic analysis through various artificial intelligence models[9-11]. Previous studies showed the accuracy and reperoducibility of landmark identification using artificial intelligence[6,10,11]. Although few studies have evaluated angular and linear measurements due to the difficulty of analyzing the reproducibility of lines and angles, evaluation of a new diagnostic system is important to assess whether it provides immediately useful results for treatment planning[12].

WebCeph (Assemblecircle, Seoul, Korea) is an orthodontic analysis platform and web-based cloud system that clinicians are available free of charge. It can save patient records, predict outcomes after orthodontic treatment, and perform cephalometric analysis using deep learning-based algorithms. WebCeph is a system developed by image training a deep learning model for adult patients with permanent dentition, and its effectiveness is reported in orthognathic surgery and adult orthodontic diagnosis[2]. However, there are insufficient studies to evaluate the accuracy of orthodontic analysis programs based on deep learning models in children and adolescents.

This study aims to validate the reliability of the application of WebCeph, a clinically acceptable deep learning-based system in adult patients, to children and adolescents without fine tuning. This study focused on whether these data were clinically useful for adolescent patients by evaluating 22 cephalometric measurements obtained through both conventional and deep learning-based methods.

This retrospective study was conducted at Ewha Womans University Mokdong Hospital. The study was approved by the Institutional Review Board (IRB) of the hospital (IRB No. 2021-03-029-003) prior to the study.

Lateral cephalometric image analysis was first introduced by Broadbent[13] in 1931. It has been used as a tool for diagnosing malocclusion, treatment planning, a growth pattern, and treatment outcome analysis. In the past, anatomical landmarks were manually traced on cephalometric radiographs. Despite being the most important procedure in cephalometric analysis, landmark identification is error-prone and time-consuming[14]. After the introduction of computer-aided analysis, cephalometric analysis using deep learning models was developed.

Deep learning algorithms use convolution filters and pooling layers to extract characteristics from images and analyze patterns. Many deep learning models have been enhanced by improving filter sizes, various ideas, and locations. The convolutional neural network, a deep neural network with multiple hidden layers, is a suitable structure for learning 2-dimensional images, such as cephalometric radiographs. These algorithms are useful for less experienced clinicians as they provide consistent results of landmark detection and measurement analysis[2]. The previous studies only investigated the accuracy of orthodontic diagnosis in patients with permanent teeth using artificial intelligence-based algorithm[6]. It is meaningful to conduct a study on the reliability of deep learning-based orthodontic analysis systems in young patients with the mixed dentition and permanent dentition.

WebCeph is a program recently developed in Korea through deep learning training of Korean adult orthodontic patients. After receiving a lateral cephalometric radiograph as input, in step 1, WebCeph detects the lateral facial region using the Support Vector Machine for the Histogram Oriented Gradients processed image. In step 2, landmarks are detected using the Ensemble of Regression Tree algorithm and the Gradient Boosting Algorithm for the detected lateral facial region. In step 3, using Faster R-CNN (ResNet), it is designed to detect a more precise location around the detected landmark location[15].

In this study, this program was chosen from among several deep learning-based programs to evaluate the accuracy of cephalometric measurements in children and adolescents. WebCeph is now free of charge, and clinicians can easily access it from mobile and computers in 22 languages. If the accuracy of the orthodontic analysis in both pediatric and adolescent patients is proven, it will help clinicians in actual clinical practice.

According to previous studies, the differences between manual and digital tracing methods were not clinically significant, and both methods were reliable[6,16]. This study evaluated the reliability by comparing deep learning-based tracing and digital tracing as a conventional method. The measurements repeated with the conventional digital cephalometric method showed high reproducibility in both groups (Table 2). This data showed the examiner could accurately reproduce measurements, and there was no difficulty in identifying the landmarks equally. These results correspond with previous studies that showed high reliability of the repeated measurements[12,17,18]. In this study, 2 repeated measurements were obtained to evaluate the reliability of the examiner based on the previous research method[12]. However, a more accurate assessment of intra-observer reliability can be obtained if repeated measurements were performed 3 times.

The influence of examiner’s clinical experience on landmark identification is significant along with the tracing method itself. In general, it is known that inter-examiner errors more frequently occur than intra-examiner errors[19]. Therefore, all measurements in this study were obtained by 1 examiner to minimize these errors. However, limitations to the reliability of the results may exist depending on a single examiner’s experience and proficiency. Thus, more reliable study results can be obtained by comparing 3 repeated measurements by 2 or more proficient examiners.

Table 3 shows the comparison results between the measurements of 2 methods by paired t-test in the mixed dentition group. The statistically significant differences between the 2 methods were detected in 8 of the 22 cephalometric variables (APDI, SNA, SNB, Mandibular plane angle, LAFH, Facial ratio, U1 to SN, and U1 to A-Pg). The comparison between the measurements of 2 methods in the permanent dentition group is shown in Table 4. In the permanent dentition group, there were no statistically significant difference between the 2 methods in most measurements, except for APDI, Wits appraisal, Facial ratio, and U1 to A-Pg.

The measurements that showed a significant difference in both groups were Facial ratio and U1 to A-Pg. Facial ratio, which is defined as the ratio percent of PFH (S - Go) and AFH (N - Me), is considered to be associated with difficulties in landmark identification itself. Differences in landmark identification used in combination of gonion and menton, which are difficult to detect, might increase the magnitude of the discrepancy. According to previous studies, the uncertainty in detecting the menton point may be caused by the difficulty of locating a landmark on curved anatomical boundaries[17]. The gonion point is defined by bisecting the angle to the tangent to the mandibular plane and the posterior border of the ramus. The detection of this landmark is difficult because it corresponds to an incorrectly defined outline related with bilateral anatomic structures and often projecting as overlapping image[12,17].

U1 to A-Pg, a variable associated with the maxillary incisors, A point and pogonion, is another measurement that showed a significant difference between 2 methods in both groups. In previous studies, it was reported that the tracing of incisor location was difficult and the incisor angular measurement was likely to change depending on the tracing methods[14,20,21]. As the A point is known to have poor reproducibility, in children with mixed dentition, locating A point may be more difficult because the image is blurred and overlaps due to the presence of tooth germs in the maxillary anterior region[12].

The number of measurements showing a statistically significant difference between the 2 methods was observed more in the mixed dentition group than in the permanent teeth group (8 measurements in the mixed dentition group, 4 measurements in the permanent dentition group). Fig. 1 is an example of the differences between WebCeph and V-Ceph program in landmark identification of a patient with mixed dentition. The radiographs traced by two methods were superimposed. Although the detection of landmarks was within the error range of 2 mm, reported as clinically acceptable in previous studies[10,11], the angular or linear measurements composed of a combination of landmarks may have a larger error due to the summation of the errors. Comparing the measurements showing a significant difference in the 2 groups, the measurements related to the identification error of nasion (N) are observed in the mixed dentition group. Previous research has indicated that it is difficult to identify nasion (N) when the nasofrontal suture is not accurately visualized, suggesting the possibility of such errors[22].

Since WebCeph is a deep learning model trained only on adult with permanent teeth, errors may occur in cephalometric measurements for pediatric and adolescent patients. It would have been difficult to accurately detect skeletal and dental measurements on cephalometric radiographs of patients with mixed dentition. The performance of a diagnostic system could be improved by fine-tuning existing deep learning models[23]. Developing a fine-tuned deep learning-based cephalometric system for patients with mixed dentition could provide more accurate analysis results.

In a previous study, Alqahtani[6] compared measurements obtained from the analysis of the computer program FACAD® (Ilexis AB, Linköping, Sweden) and that of the deep learningbased algorithm CephX® (CephX Inc., Las Vegas, USA). Alqahtani found statistically meaningful differences in 3 of 16 measurements between the 2 methods. Alqahtani reported that analysis using CephX® was fast and efficient, and the measurements obtained from both methods are correlated and highly reproducible. Similarly, this study showed statistically significant differences in 4 of 22 measurements in the permanent dentition group and 8 of 22 measurements in the mixed dentition group between the 2 methods (Table 3 and 4).

This study had the limitation of being conducted in a single center. If the study entailed collecting data from multiple institutions, more general conclusions could have been drawn. This study was conducted using a single program for cephalometric analysis in children and adolescents. Subsequent studies on the comparative evaluation of various deep learning programs for cephalometric analysis will be more meaningful. Another limitation was that the subjects of this study were selected without considering the skeletal class. Additional studies are needed for a more accurate evaluation by designing patient-classified studies according to skeletal class I, II, and III. Also, further assessment studies using larger samples will benefit the clinical applications of deep learning techniques in dentistry.

The significance of this study lies in the fact that it is the first study to study the reliability of a deep learning-based orthodontic analysis method in patients with mixed and permanent dentitions. The evaluation of deep learning-based orthodontic analysis, which provides immediately useful data for treatment planning, is meaningful for clinical applications. In the cephalometric radiographs of mixed dentition, there may be errors in tooth-related landmarks and measurements due to the overlapping of deciduous teeth and the permanent tooth germ. When performing the orthodontic analysis of children and adolescents using the deep learning-based program, it is important for dentists to be aware of the above limitations and do not rely entirely on this program. In clinical practice, dentists make individual corrections by re-positioning the landmarks after obtaining the results of orthodontic analysis from WebCeph. In using a deep learning-based analysis program as an auxiliary tool for orthodontic diagnosis of pediatric and adolescent patients, the dentist’s judgment and abundant clinical experience remain important. Understanding the above limitations, using this program appropriately can help pediatric dentists decide whether or not to begin orthodontic treatment.

Most of the measurements analyzed by the conventional and deep learning-based cephalometric system did not show a significant difference in the permanent dentition group, with the exception of some variables. However, some measurements showing a statistically significant difference between the 2 methods were more in the mixed dentition group. When using deep learning-based programs for orthodontic analysis of pediatric and adolescent patients, it is recommended to recognize the limitations of this program and use it with the proper judgment.