Introduction

Hyperbilirubinemia (HB) is a condition in newborn babies where the neonatal liver cannot clear bilirubin rapidly enough from the blood, due to an imbalance of bilirubin production and hepatic–enteric clearance.1 It is estimated that up to 60% of all newborns will experience clinical jaundice during the first week of life.2 Unfortunately, high bilirubin can lead to neurotoxicity, more specifically encephalopathy, kernicterus, and even permanent neurodevelopmental disabilities.1 HB affects an estimated 60–80% of newborn babies. It is the leading cause of re-hospitalization and is ranked seventh globally among causes of neonatal deaths in the first week of life.1 Various risk factors have been identified to predict HB, with the most prevalent factors being prematurity, hemolytic disease, perinatal infection, and exclusive breastfeeding.1

Current American Academy of Pediatrics (AAP)3 and Canadian Pediatric Society (CPS) treatment guidelines4 suggest that all newborns should be screened for total serum bilirubin (TSB) or transcutaneous bilirubin during the first 72 h of life, or earlier if they present with signs of clinical jaundice.3,5 Patients with moderate or severely elevated bilirubin levels should be treated immediately to lower the concentration of circulating bilirubin and prevent long-term neurological complications.3,5 The most widely used therapy is phototherapy, which uses light energy to non-invasively change the biochemical conformation of bilirubin to promote excretion from the body, even when normal clearance mechanisms are inadequate.2

Despite intensive treatment with phototherapy to promote clearance, up to 10% of newborns experience rebound hyperbilirubinemia (RHB), defined as bilirubin levels reaching clinical thresholds for treatment within 72 h of cessation of therapy.6,7 The main risk of discontinuing phototherapy too early is the potential for RHB, which would require re-initiation of therapy and re-hospitalization.8 In addition, current guidelines provide recommendations on when to start and stop phototherapy but do not provide information about when to check TSB levels for RHB. As a result, clinicians often delay discharge from hospital in order to obtain measurements of bilirubin levels following the termination of phototherapy to prevent potential RHB.9 Although this is a common practice, there is evidence to suggest that this practice may lead to unnecessarily prolonged hospitalization and that clinical follow-up and measurements of bilirubin levels in an outpatient setting would likely identify infants who require additional treatment.8,9 Therefore, the measurement of TSB levels following termination of phototherapy remains controversial, with many studies citing inadequate evidence to formulate recommendations surrounding post-phototherapy TSB measurements or the timeframe in which to assess patients for RHB.10

Chang et al. used stepwise logistic regression to identify predictors of RHB using a large data set of 7048 infants, with the objective of providing better clinical guidelines for clinicians surrounding the discontinuation of phototherapy and allow for accurate estimates of the probability of RHB.11 They found that RHB can be predicted with an area under the receiver operating characteristic curve (AUROC) of 0.881 (95% confidence interval (CI) 0.859–0.903) from three variables: infant’s gestational age (adjusted odds ratio (OR) of 4.7 if <38 weeks, 95% CI 3.0–7.3), age at phototherapy initiation (adjusted OR of 0.51, 95% CI 0.38–0.68), and the difference between treatment threshold values and TSB levels at phototherapy termination (adjusted OR of 1.5, 95% CI 1.4–1.7).11 More recently, in 2019, the same authors proposed a simplified prediction rule for RHB that only uses two variables: gestational age and the difference between TSB levels at phototherapy termination and the treatment threshold at phototherapy initiation.12 The authors found that a simplified score calculation was able to maintain similar discrimination as the three-variable model with an AUROC of 0.876 (95% CI 0.854–0.899).12

Despite these promising results, the use of the two- and three-variable risk prediction models have not been externally validated in other patient cohorts. Therefore, these prediction rules are of unclear clinical value and warrant further investigation. Our objectives were to assess the discrimination and calibration of these risk prediction calculations in a cohort of Canadian neonates who received phototherapy during their birth hospitalization at a tertiary care center. By doing so, we aim to validate the risk prediction tools and provide further evidence to generate clinical recommendations that can lead to effective management of RHB.

Methods

This study was approved by the Queen’s University Health Sciences and Affiliated Teaching Hospitals Research Ethics Board. The study was designed in adherence with the TRIPOD Checklist for Prediction Model Development and Validation.

Study setting and ethics approval

This study was conducted at the Kingston Health Sciences Centre (KHSC), a tertiary care teaching hospital with a 14-bed inpatient pediatrics unit (IPU) and a 24-bed neonatal intensive care unit (NICU). During the 5-year study period (January 2015 to December 2019), the average number of births at KHSC was about 2000 annually. KHSC has instituted CPS guidelines4 for the screening and treatment of HB in 2010. The risk factor assessment and treatment thresholds are similar in the CPS and AAP guidelines.3,4 Single-unit phototherapy is provided in the form of a Bili blanket on the post-partum floor to keep the mother and baby together. If indicated, infants requiring intravenous fluids or double/overhead phototherapy for treatment of HB are admitted to the IPU or NICU.

Study population

Patients include all neonates ≥35 weeks of gestation admitted to the IPU or NICU between January 2015 and December 2019 who received phototherapy for HB during their birth hospitalization (home phototherapy is not available in ON, Canada). Infants with hemolytic HB were included in the study. Preterm infants born before 35 weeks of gestation were excluded from the study, in agreement with the gestational age groups listed in the guidelines for HB3 and the exclusion criteria in the derivation studies by Chang et al.11,12 Patients whose charts were missing values for the variables used to calculate the risk prediction scores were also excluded from the study.

Data collection

Retrospective chart reviews were performed using the Patient Care System software from KHSC to identify eligible patients. One of the authors (V.S.) abstracted data from the hospital’s electronic medical records using a standardized data collection sheets to record demographic information and relevant data related to risk factors for RHB. Values related to HB treatment were extracted from the patient’s laboratory values, the medication administration record, and the nursing flowsheet. Resuscitation at birth was defined as any medical intervention (e.g., positive pressure ventilation, continuous positive airway pressure, chest compressions, or intubation) employed immediately after birth to assist babies who cannot breathe independently. Bilirubin values were converted from µmol/L from to mg/dL to ensure consistency with the calculation equation from Chang et al.,11,12 by multiplying the original value (TSB levels in µmol/L) by a conversion factor of 0.0585. The difference between threshold values and TSB measurements were obtained by subtracting the TSB measurement from the designated AAP threshold value. The relative change in TSB levels was measured by subtracting the TSB levels at treatment termination from levels at treatment initiation, for each individual patient.

If no documentation of treatment discontinuation was available in the nursing flowsheet, the time stamp of the discontinuation order was used to estimate the levels of TSB at the end of phototherapy. If documentation of treatment discontinuation was missing from the nursing flowsheet and the discontinuation order, the time of discharge from hospital was used to estimate TSB levels at the end of phototherapy. To estimate missing TSB level at the time of phototherapy termination, TSB levels closest to the time of phototherapy discontinuation were used, if within 3 h before or 3 h after treatment termination. Similar to the studies by Chang et al., if TSB levels were not measured in this time window, we estimated TSB levels at the time of phototherapy termination by linear extrapolation using the last two TSB levels before phototherapy termination, so long as the values did not include the TSB levels measured at phototherapy initiation.11,12

Risk prediction score calculations

We calculated the two- and three-variable risk prediction scores for RHB using the formulas published by Chang et al.10,11 The two-variable prediction score (TwoVarScore) = 15.5 (if gestational age <38 weeks) − 4.3 × (starting threshold − ending TSB).11 The two-variable prediction scores were converted to probabilities using the equation: 1/(1 + exp(−(0.097 × TwoVarScore − 2.84))).

The three-variable prediction score (ThreeVarScore) = 15 (if gestational age <38 weeks) − 7 × (age in days at phototherapy initiation) − 4 × (AAP phototherapy threshold − TSB at phototherapy termination) + 50.10 These scores were converted to probabilities using the equation: 1/(1 + exp(−(0.096 × ThreeVarScore − 5.19))).

Statistical analysis and performance characteristics of risk prediction scores

The data were imported into IBM SPSS Statistics for Windows, v.26 (IBM Corp., Armonk, NY). The ROC Curve command was used to obtain the AUROC curve as a measure of model discrimination. Logistic regression models were fit to estimate bivariate and multivariable ORs for putative risk factors for RHB. We checked whether the assumption of linearity of continuous variables with the logit of the outcome was met by creating a product term representing the interaction between each continuous independent variable and its natural logarithm. None of these terms was statistically significant, and therefore no transformations were applied to the continuous variables. Characteristics of the RHB and non-RHB groups were compared in SPSS and GraphPad Prism v.9 (GraphPad Software, San Diego, CA) using chi-square test or Fisher’s exact test (categorical variables), independent samples t test (normally distributed variables), or Mann–Whitney U test (non-normally distributed variables). The correlation between the two- and three-variable model scores and probabilities were compared using Pearson’s correlation coefficient (normally distributed values) or Spearman’s correlation coefficient (non-normally distributed values)

To assess model calibration, we imported the data into SAS Enterprise Guide v. 7.13 (SAS Institute Inc., Cary, NC) and used the SGPLOT procedure to fit Loess curves of predicted probabilities versus the observed outcome proportions. All references to statistical significance are based on two-tailed tests and an alpha of 0.05.

Results

Cohort characteristics

From 2015 to 2019, 305 infants with a gestational age of at least 35 weeks received phototherapy during their birth hospitalization at KHSC. Of these 305 infants, the charts of 271 neonates had the data required to calculate the risk prediction scores or had appropriate values for linear extrapolation. We used linear extrapolation to estimate missing TSB values for 4 charts (1.5%). The characteristics of the study cohort are shown in Table 1. Of the 271 neonates included in our study, 64 (23.6%) received a second treatment cycle of phototherapy (i.e., met definition of RHB). Of these 64 neonates, 55 (85.9%) reached thresholds for phototherapy at re-initiation of treatment. There was a statistically significant difference in the gestational age between the RHB and non-RHB groups (p < 0.001, Table 1).

Table 1 Cohort characteristics.
Full size table

Risk prediction scores and probability of RHB

The two-variable risk prediction scores were significantly higher (p < 0.0001, Fig. 1a) in the RHB group with a mean score (SD) of 12.67 (12.11) compared to a mean score of 5.47 (12.20) in the non-RHB group. The probability of RHB derived from the two-variable prediction scores was also significantly higher (p < 0.0001, Fig. 1c) in the RHB group with a median probability (interquartile range [IQR] of 0.146 (0.077–0.314), compared to a median probability of 0.086 (0.039–0.174) in the non-RHB group.

Fig. 1: Neonates experiencing rebound hyperbilirubinemia have higher 2-variable and 3-variable prediction scores and associated probability predictions.
figure 1

a The 2-variable risk prediction score, and b the 3-variable risk prediction score. Data represents the mean prediction calculations. Error bars represent the standard error of the mean (SEM). Each datapoint represents the risk prediction score for one retrospective chart review. c, d The probability of rebound hyperbilirubinemia derived from the 2-variable score and three-variable score, respectively. Data represents the median probability. Error bars represent the interquartile range (IQR). Each datapoint represents the probability of rebound hyperbilirubinemia for one retrospective chart review.

Full size image

Similarly, the three-variable risk prediction scores were significantly higher (p < 0.001, Fig. 1b) in the RHB group with a mean score (SD) of 40.07 (10.73), compared to a mean score of 32.33 (10.89) in the non-RHB group. The probability of RHB derived from the three-variable prediction score was also significantly higher (p < 0.0001, Fig. 1d) in the RHB group with a median probability (IQR) of 0.189 (0.109–0.366), compared to a median probability of 0.110 (0.059–0.183) in the non-RHB group. The two-variable and three-variable scores were significantly correlated (p < 0.001) with a Pearson’s coefficient of 0.846. Similarly, the probabilities of RHB derived from the two-variable and three-variable models were significantly correlated (p < 0.001) with a Spearman’s coefficient of 0.857.

Treatment of HB

The treatment of HB and RHB is summarized in Table 2. In general, the relative TSB levels remained relatively unchanged in the non-RHB group between phototherapy initiation and termination, with a mean relative change (SD) of −0.18 (2.54) mg/dL compared to 1.14 (2.20) mg/dL in the RHB group (Fig. 2a). Moreover, there was a significantly larger difference (p < 0.001) between the threshold value and measured TSB levels at phototherapy termination in the non-RHB group compared to the RHB group (seen in Fig. 2b), suggesting a larger reduction in TSB levels below the threshold value prior to phototherapy termination in the non-RHB group. More specifically, the RHB group had a mean difference (SD) of −1.88 (1.40) mg/dL compared to a difference of 2.86 (1.74) mg/dL in the non-RHB group.

Table 2 Hyperbilirubinemia treatment.
Full size table
Fig. 2: Neonates with rebound hyperbilirubinemia terminate phototherapy closer to threshold values and have increases in tsb levels during phototherapy.
figure 2

a The relative difference in total serum bilirubin (TSB) levels in mg/dL between phototherapy (PT) initiation and termination. A positive value indicates an increase in TSB levels at treatment termination relative to treatment initiation. In panel a the dashed line (- - -) represents no relative change in TSB between phototherapy initiation and termination. Each datapoint represents values from one retrospective chart review. Data represents the mean values and error bars represent the standard error (SEM) of the mean. b The difference between the threshold value for treatment and the TSB levels at treatment termination. A greater negative value suggests a larger difference between the threshold value and TSB value at treatment termination. In panel b the dashed line (- - -) represents the threshold values for phototherapy treatment. Each datapoint represents values from one retrospective chart review. Data represents the mean values and error bars represent the standard error (SEM) of the mean.

Full size image

Performance characteristics of risk prediction scores

The two- and three-variable prediction scores performed similarly, with an AUROC of 0.662 (95% CI 0.590–0.735) and 0.691 (95% CI, 0.619–0.763) respectively (Fig. 3a, b). For both the two- and three-variable prediction models, the predicted probabilities of RHB did not generally correspond to the observed proportions in our cohort (Fig. 3c, d). In general, both the two- and three-variable prediction models did not perform as well at low probability (0.0–0.3) or at higher probabilities (>0.5). The three-variable prediction score had superior calibration performance than the two-variable score.

Fig. 3: ROC curves and calibration plot for two- and three-variable prediction scores.
figure 3

a, b The receiver operating characteristic (ROC) curves for 2-variable and 3-variable risk prediction scores, respectively. Dashed lines (- - -) represent the reference line, and the solid line (—) represents the ROC curve of the prediction scores. In panel a, the area under the receiver operating characteristic (AUROC) for the 2-variable score is 0.662 (95% CI, 0.590–0.735) with standard error of 0.037. In panel b the AUROC for the 3-variable score is 0.691 (95% CI, 0.619–0.763) with a standard error of 0.037. c, d The calibration curves for 2-variable and 3-variable risk prediction scores, respectively. Dashed lines (- - -) represent perfection calibration, and the solid line (—) represents the Loess calibration curve of the prediction scores. The grey region around the solid line represents the 95% confidence interval for the calibration plot.

Full size image

Predictors of RHB

Due to the lower AUROCs for the two- and three-variable risk prediction scores in our clinical cohort than those observed in the derivation cohort,11,12 we calculated bivariate and multivariable ORs to examine which variables were the most strongly associated with RHB in our cohort compared to ORs for variables in the original derivation cohort. The results are shown in Table 3. Gestational age was strongly associated with RHB (Table 3). Variables associated with phototherapy treatment used in the three-variable prediction score, such as age of neonates at phototherapy initiation and the difference between threshold values and TSB levels at phototherapy termination, were significantly associated with RHB. In contrast, the difference between the starting threshold value and TSB levels measured at phototherapy termination (used to calculate the two-variable prediction score) was not significantly associated with RHB in the adjusted analysis (multivariable OR), although it was statistically significant for unadjusted analysis (bivariate OR).

Table 3 Bivariate and multivariable predictors of rebound hyperbilirubinemia.
Full size table

Discussion

In this study, we provide evidence that the two- and three-variable prediction scores developed by Chang et al.11,12 have unclear clinical utility in our cohort of Canadian neonates who received phototherapy during their birth hospitalization. We suspect that the observed disparities in performance characteristics are likely due to differences in the study cohorts. For example, the incidence of RHB varied substantially between our studies, with a rate of 4.6% in the studies by Chang et al.11,12 versus 23.6% in our cohort. Additionally, the cohort examined by Chang et al. included neonates who received phototherapy during their birth hospitalization (61.9% of cohort) as well as infants treated for HB after their initial birth discharge.11,12 In comparison, our study cohort only included neonates who received phototherapy during birth hospitalization, resulting in a lower mean age (SD) at phototherapy initiation in our cohort (1.5 days [0.7]) compared to the neonates in Chang et al.’s cohort (2.3 days [1.3]).11,12 This likely affects the risk of RHB, as the age of phototherapy initiation was significantly associated with RHB in the study by Chang et al.,11 as well as in other studies10 and ours (Table 3).

In addition, the cohort used by Chang et al. included neonates (4.4% of cohort) placed on home phototherapy,11,12 whereas home phototherapy is not commonly available in our practice location. As noted by Chang et al., given the unreliable estimates of TSB values associated with home phototherapy, the risk of RHB may have been underestimated in their cohort.11,12 Moreover, the cohort used by Chang et al. had a much lower rate of direct antiglobulin test (DAT) positive neonates (14.5%) compared to our cohort (31.7%), which is a well-established risk factor for the development of severe HB3 and RHB.10 However, given that our study (Table 3) did not find any significant association (p > 0.05, Tables 1 and 3) between DAT positive status and RHB, it remains unclear if DAT positive status affects the risk for RHB.

We hypothesize that the lower performance of the two- and three-variable prediction scores in our cohort may be related to the fact that around 75% of TSB levels at phototherapy termination were linearly extrapolated, using the last two TSB levels before phototherapy termination in the original studies by Chang et al.11,12 In contrast, our study only extrapolated TSB levels in 1.5% of our cohort, with most of the charts with missing TSB values being excluded (34 charts) due to a lack of suitable TSB measurements required for extrapolation. Linear extrapolation may not accurately reflect TSB levels at phototherapy termination, especially if TSB levels follow a non-linear trend. This may affect the performance of the prediction scores in external cohorts that do not utilize extrapolation for TSB values. Further investigation is required to externally validate these prediction scores in other cohorts.

Despite lower performance characteristics in our cohort compared to the derivation cohort, similar to Chang et al., we found that gestational age was strongly associated with RHB (seen in Table 3). We also found that the other variables included in the three-variable risk prediction, namely, the age of neonates at phototherapy initiation and the difference between threshold values and TSB levels at phototherapy termination, were significantly associated with RHB. The results from our study, as well as Chang et al.’s study, differ from a previous study that showed no statistical difference in the rates of RHB between neonates with small differences (≥1 mg/dL) and large differences (≥3 mg/dL) between the treatment threshold and TSB levels at phototherapy termination, although this study was underpowered to detect such differences given the small sample size (n = 52).13

In contrast, although all of the factors in the three-variable prediction score were significantly associated with RHB, this was not the case for both variables in the two-variable prediction model: the difference between the starting threshold value and TSB measured at phototherapy termination (from the two-variable score) was not significantly associated with RHB (in adjusted analysis using multivariable OR). This may be one reason for the poorer calibration of the two-variable model compared to the three-variable model. Taken together, we suggest that the three-variable prediction model has better performance than the two-variable prediction model, despite their similar AUROCs.

Given the lower performance characteristics of the two- and three-variable prediction models in our cohort of neonates who received phototherapy during their birth hospitalization, the clinical utility of these tools is currently unclear. Thus, these prediction scores and probability estimates should be interpreted with caution until follow-up studies provide additional validation. Nevertheless, our results suggest that RHB is significantly associated with the following variables: (1) gestational age (<40 weeks), (2) age at phototherapy start, and (3) the difference between threshold values and measured TSB levels at termination.

A major limitation of our study was the small sample size of 271 neonates. This resulted in imprecise estimates of model performance, although we note that the upper limit of the 95% CIs for our AUROCs were still substantially lower than the AUROCs reported by Chang et al.11,12 In addition, similar to studies by Chang et al.,11,12 we did not evaluate the risk of RHB after a second episode of inpatient phototherapy, owing to the small proportion of neonates experiencing RHB (N = 64) in our cohort. Future studies evaluating births over a longer time frame, and/or from multiple sites, are critical to validate and identify additional risk factors for RHB to modify existing risk prediction models. In addition, our study was limited by the retrospective chart review design, which led to an inability to capture other established risk factors for neonatal HB, such as maternal race/ethnicity.14 Future studies could prospectively collect data to further evaluate whether such factors, which are often missing from neonatal records, affect the risk of RHB.

In summary, our study provides insufficient evidence to support the use of published two- and three-variable risk prediction scores for neonates who require phototherapy during their birth hospitalization. More studies are required to further validate these prediction scores.