22/11/29 12:40:58 WARN XGBoostSpark: train_test_ratio is deprecated since XGBoost 0.82, we recommend to explicitly pass a training and multiple evaluation datasets by passing 'eval_sets' and 'eval_set_names' Tracker started, with env={DMLC_NUM_SERVER=0, DMLC_TRACKER_URI=10.10.0.69, DMLC_TRACKER_PORT=9091, DMLC_NUM_WORKER=4} 22/11/29 12:40:58 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU *Exec will run on GPU *Expression month(monthly_reporting_period#443) AS timestamp_month#1245 will run on GPU *Expression month(monthly_reporting_period#443) will run on GPU *Expression year(monthly_reporting_period#443) AS timestamp_year#1281 will run on GPU *Expression year(monthly_reporting_period#443) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#216 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_year#1171L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_month#1200L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) AS delinquency_12#1217 will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) will run on GPU *Expression cast((delinquency_12#1156 > 3) as int) will run on GPU *Expression (delinquency_12#1156 > 3) will run on GPU *Expression cast((upb_12#1158 = 0.0) as int) will run on GPU *Expression (upb_12#1158 = 0.0) will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END AS timestamp_month#1200L will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END will run on GPU *Expression (timestamp_month_tmp#1185L = 0) will run on GPU *Exec will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression max(delinquency_12#909)#1155 AS delinquency_12#1156 will run on GPU *Expression min(upb_12#945)#1157 AS upb_12#1158 will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) AS timestamp_year#1171L will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) AS timestamp_month_tmp#1185L will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression partial_max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression partial_min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Exec will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) AS josh_mody_n#1128L will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Exec will run on GPU *Expression (isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) AND ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)))) will run on GPU *Expression isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) will run on GPU *Expression FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12))) will run on GPU *Expression (pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Expression isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression explode([0,1,2,3,4,5,6,7,8,9,10,11]) will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression current_loan_delinquency_status#1356 AS delinquency_12#909 will run on GPU *Expression current_actual_upb#1328 AS upb_12#945 will run on GPU *Expression month(monthly_reporting_period#443) AS timestamp_month#837 will run on GPU *Expression month(monthly_reporting_period#443) will run on GPU *Expression year(monthly_reporting_period#443) AS timestamp_year#873 will run on GPU *Expression year(monthly_reporting_period#443) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1318L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1425 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1096 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#989L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Expression (delinquency_12#777 >= 1) AS ever_30#790 will run on GPU *Expression (delinquency_12#777 >= 1) will run on GPU *Expression (delinquency_12#777 >= 3) AS ever_90#791 will run on GPU *Expression (delinquency_12#777 >= 3) will run on GPU *Expression (delinquency_12#777 >= 6) AS ever_180#792 will run on GPU *Expression (delinquency_12#777 >= 6) will run on GPU *Exec will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression max(current_loan_delinquency_status#1027)#776 AS delinquency_12#777 will run on GPU *Expression min(delinquency_30#761)#778 AS delinquency_30#779 will run on GPU *Expression min(delinquency_90#762)#780 AS delinquency_90#781 will run on GPU *Expression min(delinquency_180#763)#782 AS delinquency_180#783 will run on GPU *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression partial_max(current_loan_delinquency_status#1027) will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression partial_min(delinquency_30#761) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression partial_min(delinquency_90#762) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression partial_min(delinquency_180#763) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Exec will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 1) THEN monthly_reporting_period#443 END AS delinquency_30#761 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 1) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 1) will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 3) THEN monthly_reporting_period#443 END AS delinquency_90#762 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 3) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 3) will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 6) THEN monthly_reporting_period#443 END AS delinquency_180#763 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 6) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 6) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#989L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1096 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) AS seller_name#1631 will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression seller_name#327 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression (rank#332 = 1) will run on GPU *Exec will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#332 will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression dense_rank(_w1#343) will run on GPU *Expression windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Expression specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$()) will run on GPU *Expression unboundedpreceding$() will run on GPU *Expression currentrow$() will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression loan_id#1748L ASC NULLS FIRST will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression upper(seller_name#1751) AS seller_name#327 will run on GPU *Expression upper(seller_name#1751) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) AS _w1#343 will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1748L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1855 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression from_seller_name#1511 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression isnotnull(from_seller_name#1511) will run on GPU ! cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.execution.RDDScanExec @Expression from_seller_name#1511 could run on GPU @Expression to_seller_name#1512 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS orig_channel#2066 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 0) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 0) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 0) could run on GPU @Expression column_id#1997 could run on GPU @Expression 0 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS first_home_buyer#2712 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 1) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 1) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 1) could run on GPU @Expression column_id#1997 could run on GPU @Expression 1 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS loan_purpose#3358 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 2) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 2) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 2) could run on GPU @Expression column_id#1997 could run on GPU @Expression 2 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS property_type#4004 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 3) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 3) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 3) could run on GPU @Expression column_id#1997 could run on GPU @Expression 3 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS occupancy_status#4650 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 4) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 4) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 4) could run on GPU @Expression column_id#1997 could run on GPU @Expression 4 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS property_state#5296 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 5) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 5) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 5) could run on GPU @Expression column_id#1997 could run on GPU @Expression 5 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS product_type#5942 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 6) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 6) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 6) could run on GPU @Expression column_id#1997 could run on GPU @Expression 6 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS relocation_mortgage_indicator#6588 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 7) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 7) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 7) could run on GPU @Expression column_id#1997 could run on GPU @Expression 7 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS seller_name#7234 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 8) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 8) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 8) could run on GPU @Expression column_id#1997 could run on GPU @Expression 8 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS mod_flag#7880 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 9) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 9) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 9) could run on GPU @Expression column_id#1997 could run on GPU @Expression 9 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU *Exec will run on GPU *Expression month(monthly_reporting_period#443) AS timestamp_month#1245 will run on GPU *Expression month(monthly_reporting_period#443) will run on GPU *Expression year(monthly_reporting_period#443) AS timestamp_year#1281 will run on GPU *Expression year(monthly_reporting_period#443) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#216 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_year#1171L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_month#1200L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) AS delinquency_12#1217 will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) will run on GPU *Expression cast((delinquency_12#1156 > 3) as int) will run on GPU *Expression (delinquency_12#1156 > 3) will run on GPU *Expression cast((upb_12#1158 = 0.0) as int) will run on GPU *Expression (upb_12#1158 = 0.0) will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END AS timestamp_month#1200L will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END will run on GPU *Expression (timestamp_month_tmp#1185L = 0) will run on GPU *Exec will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression max(delinquency_12#909)#1155 AS delinquency_12#1156 will run on GPU *Expression min(upb_12#945)#1157 AS upb_12#1158 will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) AS timestamp_year#1171L will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) AS timestamp_month_tmp#1185L will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression partial_max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression partial_min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Exec will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) AS josh_mody_n#1128L will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Exec will run on GPU *Expression (isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) AND ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)))) will run on GPU *Expression isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) will run on GPU *Expression FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12))) will run on GPU *Expression (pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Expression isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression explode([0,1,2,3,4,5,6,7,8,9,10,11]) will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression current_loan_delinquency_status#1356 AS delinquency_12#909 will run on GPU *Expression current_actual_upb#1328 AS upb_12#945 will run on GPU *Expression month(monthly_reporting_period#443) AS timestamp_month#837 will run on GPU *Expression month(monthly_reporting_period#443) will run on GPU *Expression year(monthly_reporting_period#443) AS timestamp_year#873 will run on GPU *Expression year(monthly_reporting_period#443) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1318L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1425 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1096 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#989L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Expression (delinquency_12#777 >= 1) AS ever_30#790 will run on GPU *Expression (delinquency_12#777 >= 1) will run on GPU *Expression (delinquency_12#777 >= 3) AS ever_90#791 will run on GPU *Expression (delinquency_12#777 >= 3) will run on GPU *Expression (delinquency_12#777 >= 6) AS ever_180#792 will run on GPU *Expression (delinquency_12#777 >= 6) will run on GPU *Exec will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression max(current_loan_delinquency_status#1027)#776 AS delinquency_12#777 will run on GPU *Expression min(delinquency_30#761)#778 AS delinquency_30#779 will run on GPU *Expression min(delinquency_90#762)#780 AS delinquency_90#781 will run on GPU *Expression min(delinquency_180#763)#782 AS delinquency_180#783 will run on GPU *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression partial_max(current_loan_delinquency_status#1027) will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression partial_min(delinquency_30#761) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression partial_min(delinquency_90#762) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression partial_min(delinquency_180#763) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Exec will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 1) THEN monthly_reporting_period#443 END AS delinquency_30#761 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 1) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 1) will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 3) THEN monthly_reporting_period#443 END AS delinquency_90#762 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 3) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 3) will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 6) THEN monthly_reporting_period#443 END AS delinquency_180#763 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 6) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 6) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#989L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1096 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) AS seller_name#1631 will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression seller_name#327 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression (rank#332 = 1) will run on GPU *Exec will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#332 will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression dense_rank(_w1#343) will run on GPU *Expression windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Expression specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$()) will run on GPU *Expression unboundedpreceding$() will run on GPU *Expression currentrow$() will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression loan_id#1748L ASC NULLS FIRST will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression upper(seller_name#1751) AS seller_name#327 will run on GPU *Expression upper(seller_name#1751) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) AS _w1#343 will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1748L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1855 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression from_seller_name#1511 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression isnotnull(from_seller_name#1511) will run on GPU ! cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.execution.RDDScanExec @Expression from_seller_name#1511 could run on GPU @Expression to_seller_name#1512 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS orig_channel#2066 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 0) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 0) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 0) could run on GPU @Expression column_id#1997 could run on GPU @Expression 0 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS first_home_buyer#2712 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 1) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 1) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 1) could run on GPU @Expression column_id#1997 could run on GPU @Expression 1 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS loan_purpose#3358 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 2) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 2) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 2) could run on GPU @Expression column_id#1997 could run on GPU @Expression 2 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS property_type#4004 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 3) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 3) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 3) could run on GPU @Expression column_id#1997 could run on GPU @Expression 3 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS occupancy_status#4650 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 4) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 4) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 4) could run on GPU @Expression column_id#1997 could run on GPU @Expression 4 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS property_state#5296 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 5) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 5) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 5) could run on GPU @Expression column_id#1997 could run on GPU @Expression 5 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS product_type#5942 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 6) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 6) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 6) could run on GPU @Expression column_id#1997 could run on GPU @Expression 6 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS relocation_mortgage_indicator#6588 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 7) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 7) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 7) could run on GPU @Expression column_id#1997 could run on GPU @Expression 7 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS seller_name#7234 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 8) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 8) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 8) could run on GPU @Expression column_id#1997 could run on GPU @Expression 8 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS mod_flag#7880 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 9) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 9) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 9) could run on GPU @Expression column_id#1997 could run on GPU @Expression 9 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU *Exec will run on GPU *Expression month(monthly_reporting_period#443) AS timestamp_month#1245 will run on GPU *Expression month(monthly_reporting_period#443) will run on GPU *Expression year(monthly_reporting_period#443) AS timestamp_year#1281 will run on GPU *Expression year(monthly_reporting_period#443) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#216 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_year#1171L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_month#1200L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) AS delinquency_12#1217 will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) will run on GPU *Expression cast((delinquency_12#1156 > 3) as int) will run on GPU *Expression (delinquency_12#1156 > 3) will run on GPU *Expression cast((upb_12#1158 = 0.0) as int) will run on GPU *Expression (upb_12#1158 = 0.0) will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END AS timestamp_month#1200L will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END will run on GPU *Expression (timestamp_month_tmp#1185L = 0) will run on GPU *Exec will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression max(delinquency_12#909)#1155 AS delinquency_12#1156 will run on GPU *Expression min(upb_12#945)#1157 AS upb_12#1158 will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) AS timestamp_year#1171L will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) AS timestamp_month_tmp#1185L will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression partial_max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression partial_min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Exec will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) AS josh_mody_n#1128L will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Exec will run on GPU *Expression (isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) AND ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)))) will run on GPU *Expression isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) will run on GPU *Expression FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12))) will run on GPU *Expression (pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Expression isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression explode([0,1,2,3,4,5,6,7,8,9,10,11]) will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression current_loan_delinquency_status#1356 AS delinquency_12#909 will run on GPU *Expression current_actual_upb#1328 AS upb_12#945 will run on GPU *Expression month(monthly_reporting_period#443) AS timestamp_month#837 will run on GPU *Expression month(monthly_reporting_period#443) will run on GPU *Expression year(monthly_reporting_period#443) AS timestamp_year#873 will run on GPU *Expression year(monthly_reporting_period#443) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1318L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1425 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1096 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#989L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Expression (delinquency_12#777 >= 1) AS ever_30#790 will run on GPU *Expression (delinquency_12#777 >= 1) will run on GPU *Expression (delinquency_12#777 >= 3) AS ever_90#791 will run on GPU *Expression (delinquency_12#777 >= 3) will run on GPU *Expression (delinquency_12#777 >= 6) AS ever_180#792 will run on GPU *Expression (delinquency_12#777 >= 6) will run on GPU *Exec will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression max(current_loan_delinquency_status#1027)#776 AS delinquency_12#777 will run on GPU *Expression min(delinquency_30#761)#778 AS delinquency_30#779 will run on GPU *Expression min(delinquency_90#762)#780 AS delinquency_90#781 will run on GPU *Expression min(delinquency_180#763)#782 AS delinquency_180#783 will run on GPU *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression partial_max(current_loan_delinquency_status#1027) will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression partial_min(delinquency_30#761) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression partial_min(delinquency_90#762) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression partial_min(delinquency_180#763) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Exec will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 1) THEN monthly_reporting_period#443 END AS delinquency_30#761 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 1) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 1) will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 3) THEN monthly_reporting_period#443 END AS delinquency_90#762 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 3) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 3) will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 6) THEN monthly_reporting_period#443 END AS delinquency_180#763 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 6) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 6) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#989L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1096 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) AS seller_name#1631 will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression seller_name#327 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression (rank#332 = 1) will run on GPU *Exec will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#332 will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression dense_rank(_w1#343) will run on GPU *Expression windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Expression specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$()) will run on GPU *Expression unboundedpreceding$() will run on GPU *Expression currentrow$() will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression loan_id#1748L ASC NULLS FIRST will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression upper(seller_name#1751) AS seller_name#327 will run on GPU *Expression upper(seller_name#1751) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) AS _w1#343 will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1748L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1855 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression from_seller_name#1511 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression isnotnull(from_seller_name#1511) will run on GPU ! cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.execution.RDDScanExec @Expression from_seller_name#1511 could run on GPU @Expression to_seller_name#1512 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS orig_channel#2066 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 0) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 0) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 0) could run on GPU @Expression column_id#1997 could run on GPU @Expression 0 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS first_home_buyer#2712 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 1) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 1) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 1) could run on GPU @Expression column_id#1997 could run on GPU @Expression 1 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS loan_purpose#3358 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 2) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 2) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 2) could run on GPU @Expression column_id#1997 could run on GPU @Expression 2 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS property_type#4004 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 3) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 3) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 3) could run on GPU @Expression column_id#1997 could run on GPU @Expression 3 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS occupancy_status#4650 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 4) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 4) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 4) could run on GPU @Expression column_id#1997 could run on GPU @Expression 4 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS property_state#5296 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 5) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 5) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 5) could run on GPU @Expression column_id#1997 could run on GPU @Expression 5 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS product_type#5942 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 6) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 6) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 6) could run on GPU @Expression column_id#1997 could run on GPU @Expression 6 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS relocation_mortgage_indicator#6588 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 7) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 7) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 7) could run on GPU @Expression column_id#1997 could run on GPU @Expression 7 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS seller_name#7234 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 8) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 8) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 8) could run on GPU @Expression column_id#1997 could run on GPU @Expression 8 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS mod_flag#7880 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 9) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 9) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 9) could run on GPU @Expression column_id#1997 could run on GPU @Expression 9 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Partitioning will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU *Exec will run on GPU *Expression month(monthly_reporting_period#443) AS timestamp_month#1245 will run on GPU *Expression month(monthly_reporting_period#443) will run on GPU *Expression year(monthly_reporting_period#443) AS timestamp_year#1281 will run on GPU *Expression year(monthly_reporting_period#443) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#2, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#216 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression current_loan_delinquency_status#1356 AS delinquency_12#909 will run on GPU *Expression current_actual_upb#1328 AS upb_12#945 will run on GPU *Expression month(monthly_reporting_period#443) AS timestamp_month#837 will run on GPU *Expression month(monthly_reporting_period#443) will run on GPU *Expression year(monthly_reporting_period#443) AS timestamp_year#873 will run on GPU *Expression year(monthly_reporting_period#443) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#1319, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1318L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1425 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression partial_max(current_loan_delinquency_status#1027) will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression partial_min(delinquency_30#761) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression partial_min(delinquency_90#762) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression partial_min(delinquency_180#763) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Exec will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 1) THEN monthly_reporting_period#443 END AS delinquency_30#761 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 1) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 1) will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 3) THEN monthly_reporting_period#443 END AS delinquency_90#762 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 3) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 3) will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 6) THEN monthly_reporting_period#443 END AS delinquency_180#763 will run on GPU *Expression CASE WHEN (current_loan_delinquency_status#1027 >= 6) THEN monthly_reporting_period#443 END will run on GPU *Expression (current_loan_delinquency_status#1027 >= 6) will run on GPU *Exec will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) AS monthly_reporting_period#443 will run on GPU *Expression cast(gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)), MM/dd/yyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Expression date_format(cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp), MM/dd/yyyy, Some(Etc/UTC)) will run on GPU *Expression cast(cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) as timestamp) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#990, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#989L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1096 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression upper(seller_name#1751) AS seller_name#327 will run on GPU *Expression upper(seller_name#1751) will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) AS _w1#343 will run on GPU *Expression cast(gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) as date) will run on GPU *Expression gettimestamp(monthly_reporting_period#1749, MMyyyy, TimestampType, Some(Etc/UTC), false) will run on GPU *Exec will run on GPU *Expression isnotnull(loan_id#1748L) will run on GPU *Exec will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) AS quarter#1855 will run on GPU *Expression substring_index(substring_index(input_file_name(), ., 1), _, -1) will run on GPU *Expression substring_index(input_file_name(), ., 1) will run on GPU *Expression input_file_name() will run on GPU *Exec will run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression isnotnull(from_seller_name#1511) will run on GPU ! cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.execution.RDDScanExec @Expression from_seller_name#1511 could run on GPU @Expression to_seller_name#1512 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS orig_channel#2066 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 0) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 0) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 0) could run on GPU @Expression column_id#1997 could run on GPU @Expression 0 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS first_home_buyer#2712 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 1) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 1) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 1) could run on GPU @Expression column_id#1997 could run on GPU @Expression 1 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS loan_purpose#3358 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 2) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 2) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 2) could run on GPU @Expression column_id#1997 could run on GPU @Expression 2 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS property_type#4004 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 3) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 3) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 3) could run on GPU @Expression column_id#1997 could run on GPU @Expression 3 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS occupancy_status#4650 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 4) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 4) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 4) could run on GPU @Expression column_id#1997 could run on GPU @Expression 4 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS property_state#5296 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 5) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 5) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 5) could run on GPU @Expression column_id#1997 could run on GPU @Expression 5 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS product_type#5942 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 6) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 6) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 6) could run on GPU @Expression column_id#1997 could run on GPU @Expression 6 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS relocation_mortgage_indicator#6588 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 7) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 7) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 7) could run on GPU @Expression column_id#1997 could run on GPU @Expression 7 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS seller_name#7234 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 8) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 8) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 8) could run on GPU @Expression column_id#1997 could run on GPU @Expression 8 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU 22/11/29 12:40:59 WARN GpuOverrides: *Exec will run on GPU *Exec will run on GPU *Expression data#2000 AS mod_flag#7880 will run on GPU *Exec will run on GPU *Expression ((column_id#1997 = 9) AND isnotnull(data#2000)) will run on GPU *Expression (column_id#1997 = 9) will run on GPU *Expression isnotnull(data#2000) will run on GPU !Exec cannot run on GPU because ParquetCachedBatchSerializer is not being used @Expression column_id#1997 could run on GPU @Expression data#2000 could run on GPU @Expression id#2011 could run on GPU @Expression (column_id#1997 = 9) could run on GPU @Expression column_id#1997 could run on GPU @Expression 9 could run on GPU @Expression isnotnull(data#2000) could run on GPU @Expression data#2000 could run on GPU [Stage 1:==>(12 + 6) / 18][Stage 2:> (0 + 2) / 18][Stage 3:> (0 + 0) / 18]8] 22/11/29 12:41:18 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_year#1171L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_month#1200L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) AS delinquency_12#1217 will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) will run on GPU *Expression cast((delinquency_12#1156 > 3) as int) will run on GPU *Expression (delinquency_12#1156 > 3) will run on GPU *Expression cast((upb_12#1158 = 0.0) as int) will run on GPU *Expression (upb_12#1158 = 0.0) will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END AS timestamp_month#1200L will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END will run on GPU *Expression (timestamp_month_tmp#1185L = 0) will run on GPU *Exec will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression max(delinquency_12#909)#1155 AS delinquency_12#1156 will run on GPU *Expression min(upb_12#945)#1157 AS upb_12#1158 will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) AS timestamp_year#1171L will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) AS timestamp_month_tmp#1185L will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression partial_max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression partial_min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Exec will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) AS josh_mody_n#1128L will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Exec will run on GPU *Expression (isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) AND ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)))) will run on GPU *Expression isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) will run on GPU *Expression FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12))) will run on GPU *Expression (pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Expression isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression explode([0,1,2,3,4,5,6,7,8,9,10,11]) will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1096 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#989L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Expression (delinquency_12#777 >= 1) AS ever_30#790 will run on GPU *Expression (delinquency_12#777 >= 1) will run on GPU *Expression (delinquency_12#777 >= 3) AS ever_90#791 will run on GPU *Expression (delinquency_12#777 >= 3) will run on GPU *Expression (delinquency_12#777 >= 6) AS ever_180#792 will run on GPU *Expression (delinquency_12#777 >= 6) will run on GPU *Exec will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression max(current_loan_delinquency_status#1027)#776 AS delinquency_12#777 will run on GPU *Expression min(delinquency_30#761)#778 AS delinquency_30#779 will run on GPU *Expression min(delinquency_90#762)#780 AS delinquency_90#781 will run on GPU *Expression min(delinquency_180#763)#782 AS delinquency_180#783 will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) AS seller_name#1631 will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression seller_name#327 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression (rank#332 = 1) will run on GPU *Exec will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#332 will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression dense_rank(_w1#343) will run on GPU *Expression windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Expression specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$()) will run on GPU *Expression unboundedpreceding$() will run on GPU *Expression currentrow$() will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression loan_id#1748L ASC NULLS FIRST will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression from_seller_name#1511 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed [Stage 2:> (2 + 8) / 18][Stage 3:> (0 + 0) / 18][Stage 4:> (0 + 0) / 8] 22/11/29 12:41:19 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_year#1171L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_month#1200L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) AS delinquency_12#1217 will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) will run on GPU *Expression cast((delinquency_12#1156 > 3) as int) will run on GPU *Expression (delinquency_12#1156 > 3) will run on GPU *Expression cast((upb_12#1158 = 0.0) as int) will run on GPU *Expression (upb_12#1158 = 0.0) will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END AS timestamp_month#1200L will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END will run on GPU *Expression (timestamp_month_tmp#1185L = 0) will run on GPU *Exec will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression max(delinquency_12#909)#1155 AS delinquency_12#1156 will run on GPU *Expression min(upb_12#945)#1157 AS upb_12#1158 will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) AS timestamp_year#1171L will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) AS timestamp_month_tmp#1185L will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression partial_max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression partial_min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Exec will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) AS josh_mody_n#1128L will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Exec will run on GPU *Expression (isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) AND ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)))) will run on GPU *Expression isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) will run on GPU *Expression FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12))) will run on GPU *Expression (pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Expression isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression explode([0,1,2,3,4,5,6,7,8,9,10,11]) will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1096 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#989L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Expression (delinquency_12#777 >= 1) AS ever_30#790 will run on GPU *Expression (delinquency_12#777 >= 1) will run on GPU *Expression (delinquency_12#777 >= 3) AS ever_90#791 will run on GPU *Expression (delinquency_12#777 >= 3) will run on GPU *Expression (delinquency_12#777 >= 6) AS ever_180#792 will run on GPU *Expression (delinquency_12#777 >= 6) will run on GPU *Exec will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression max(current_loan_delinquency_status#1027)#776 AS delinquency_12#777 will run on GPU *Expression min(delinquency_30#761)#778 AS delinquency_30#779 will run on GPU *Expression min(delinquency_90#762)#780 AS delinquency_90#781 will run on GPU *Expression min(delinquency_180#763)#782 AS delinquency_180#783 will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) AS seller_name#1631 will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression seller_name#327 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression (rank#332 = 1) will run on GPU *Exec will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#332 will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression dense_rank(_w1#343) will run on GPU *Expression windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Expression specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$()) will run on GPU *Expression unboundedpreceding$() will run on GPU *Expression currentrow$() will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression loan_id#1748L ASC NULLS FIRST will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression from_seller_name#1511 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed [Stage 3:> (0 + 8) / 18][Stage 4:> (0 + 0) / 8][Stage 5:> (0 + 0) / 18] 22/11/29 12:41:20 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_year#1171L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_month#1200L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) AS delinquency_12#1217 will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) will run on GPU *Expression cast((delinquency_12#1156 > 3) as int) will run on GPU *Expression (delinquency_12#1156 > 3) will run on GPU *Expression cast((upb_12#1158 = 0.0) as int) will run on GPU *Expression (upb_12#1158 = 0.0) will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END AS timestamp_month#1200L will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END will run on GPU *Expression (timestamp_month_tmp#1185L = 0) will run on GPU *Exec will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression max(delinquency_12#909)#1155 AS delinquency_12#1156 will run on GPU *Expression min(upb_12#945)#1157 AS upb_12#1158 will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) AS timestamp_year#1171L will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) AS timestamp_month_tmp#1185L will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression partial_max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression partial_min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Exec will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) AS josh_mody_n#1128L will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Exec will run on GPU *Expression (isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) AND ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)))) will run on GPU *Expression isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) will run on GPU *Expression FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12))) will run on GPU *Expression (pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Expression isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression explode([0,1,2,3,4,5,6,7,8,9,10,11]) will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1096 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#989L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Expression (delinquency_12#777 >= 1) AS ever_30#790 will run on GPU *Expression (delinquency_12#777 >= 1) will run on GPU *Expression (delinquency_12#777 >= 3) AS ever_90#791 will run on GPU *Expression (delinquency_12#777 >= 3) will run on GPU *Expression (delinquency_12#777 >= 6) AS ever_180#792 will run on GPU *Expression (delinquency_12#777 >= 6) will run on GPU *Exec will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression max(current_loan_delinquency_status#1027)#776 AS delinquency_12#777 will run on GPU *Expression min(delinquency_30#761)#778 AS delinquency_30#779 will run on GPU *Expression min(delinquency_90#762)#780 AS delinquency_90#781 will run on GPU *Expression min(delinquency_180#763)#782 AS delinquency_180#783 will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) AS seller_name#1631 will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression seller_name#327 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression (rank#332 = 1) will run on GPU *Exec will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#332 will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression dense_rank(_w1#343) will run on GPU *Expression windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Expression specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$()) will run on GPU *Expression unboundedpreceding$() will run on GPU *Expression currentrow$() will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression loan_id#1748L ASC NULLS FIRST will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression from_seller_name#1511 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:20 WARN GpuOverrides: *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) AS delinquency_12#1217 will run on GPU *Expression (cast((delinquency_12#1156 > 3) as int) + cast((upb_12#1158 = 0.0) as int)) will run on GPU *Expression cast((delinquency_12#1156 > 3) as int) will run on GPU *Expression (delinquency_12#1156 > 3) will run on GPU *Expression cast((upb_12#1158 = 0.0) as int) will run on GPU *Expression (upb_12#1158 = 0.0) will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END AS timestamp_month#1200L will run on GPU *Expression CASE WHEN (timestamp_month_tmp#1185L = 0) THEN 12 ELSE timestamp_month_tmp#1185L END will run on GPU *Expression (timestamp_month_tmp#1185L = 0) will run on GPU *Exec will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Expression max(delinquency_12#909)#1155 AS delinquency_12#1156 will run on GPU *Expression min(upb_12#945)#1157 AS upb_12#1158 will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) AS timestamp_year#1171L will run on GPU *Expression FLOOR((cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) AS timestamp_month_tmp#1185L will run on GPU *Expression pmod(((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (josh_mody_n#1128L * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (josh_mody_n#1128L * 12)) will run on GPU *Expression (josh_mody_n#1128L * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression partial_max(delinquency_12#909) will run on GPU *Expression max(delinquency_12#909) will run on GPU *Expression partial_min(upb_12#945) will run on GPU *Expression min(upb_12#945) will run on GPU *Exec will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) AS josh_mody_n#1128L will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Exec will run on GPU *Expression (isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) AND ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)))) will run on GPU *Expression isnotnull(FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0))) will run on GPU *Expression FLOOR((cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0)) will run on GPU *Expression (cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) / 12.0) will run on GPU *Expression cast(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) as double) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast((month_y#1112 - 1) as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast((month_y#1112 - 1) as bigint) will run on GPU *Expression (month_y#1112 - 1) will run on GPU *Expression ((pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) OR isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12))) will run on GPU *Expression (pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) = 0) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Expression isnotnull(pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12)) will run on GPU *Expression pmod(((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)), 12) will run on GPU *Expression ((24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) + cast(month_y#1112 as bigint)) will run on GPU *Expression (24000 + (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12)) will run on GPU *Expression (FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) * 12) will run on GPU *Expression FLOOR((cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0)) will run on GPU *Expression (cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) / 12.0) will run on GPU *Expression cast(((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) as double) will run on GPU *Expression ((((timestamp_year#873 * 12) + timestamp_month#837) - 24000) - month_y#1112) will run on GPU *Expression (((timestamp_year#873 * 12) + timestamp_month#837) - 24000) will run on GPU *Expression ((timestamp_year#873 * 12) + timestamp_month#837) will run on GPU *Expression (timestamp_year#873 * 12) will run on GPU *Expression cast(month_y#1112 as bigint) will run on GPU *Exec will run on GPU *Expression explode([0,1,2,3,4,5,6,7,8,9,10,11]) will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1096 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#989L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Expression (delinquency_12#777 >= 1) AS ever_30#790 will run on GPU *Expression (delinquency_12#777 >= 1) will run on GPU *Expression (delinquency_12#777 >= 3) AS ever_90#791 will run on GPU *Expression (delinquency_12#777 >= 3) will run on GPU *Expression (delinquency_12#777 >= 6) AS ever_180#792 will run on GPU *Expression (delinquency_12#777 >= 6) will run on GPU *Exec will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression max(current_loan_delinquency_status#1027) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_30#761) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_90#762) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression min(delinquency_180#763) will run on GPU *Expression max(current_loan_delinquency_status#1027)#776 AS delinquency_12#777 will run on GPU *Expression min(delinquency_30#761)#778 AS delinquency_30#779 will run on GPU *Expression min(delinquency_90#762)#780 AS delinquency_90#781 will run on GPU *Expression min(delinquency_180#763)#782 AS delinquency_180#783 will run on GPU *Exec will run on GPU [Stage 3:==>(17 + 1) / 18][Stage 5:> (0 + 7) / 18][Stage 7:> (0 + 0) / 8] 22/11/29 12:41:23 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_year#1171L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_month#1200L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) AS seller_name#1631 will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression (rank#332 = 1) will run on GPU *Exec will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#332 will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression dense_rank(_w1#343) will run on GPU *Expression windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Expression specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$()) will run on GPU *Expression unboundedpreceding$() will run on GPU *Expression currentrow$() will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression loan_id#1748L ASC NULLS FIRST will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU 22/11/29 12:41:23 WARN GpuOverrides: *Exec will run on GPU *Exec will run on GPU 22/11/29 12:41:23 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_year#1171L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_month#1200L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) AS seller_name#1631 will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression (rank#332 = 1) will run on GPU *Exec will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#332 will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression dense_rank(_w1#343) will run on GPU *Expression windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Expression specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$()) will run on GPU *Expression unboundedpreceding$() will run on GPU *Expression currentrow$() will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression loan_id#1748L ASC NULLS FIRST will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU 22/11/29 12:41:44 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_year#1171L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_month#1200L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) AS seller_name#1631 will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression (rank#332 = 1) will run on GPU *Exec will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#332 will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression dense_rank(_w1#343) will run on GPU *Expression windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Expression specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$()) will run on GPU *Expression unboundedpreceding$() will run on GPU *Expression currentrow$() will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression loan_id#1748L ASC NULLS FIRST will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU 22/11/29 12:41:44 WARN GpuOverrides: *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) AS seller_name#1631 will run on GPU *Expression coalesce(to_seller_name#1512, seller_name#327) will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression (rank#332 = 1) will run on GPU *Exec will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS rank#332 will run on GPU *Expression dense_rank(_w1#343) windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression dense_rank(_w1#343) will run on GPU *Expression windowspecdefinition(loan_id#1748L, _w1#343 ASC NULLS FIRST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Expression specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$()) will run on GPU *Expression unboundedpreceding$() will run on GPU *Expression currentrow$() will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression loan_id#1748L ASC NULLS FIRST will run on GPU *Expression _w1#343 ASC NULLS FIRST will run on GPU *Exec will run on GPU 22/11/29 12:41:44 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_year#1171L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_month#1200L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:44 WARN GpuOverrides: *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression cast(timestamp_year#1281 as bigint) will run on GPU *Expression cast(timestamp_month#1245 as bigint) will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_year#1281 as bigint) could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression cast(timestamp_month#1245 as bigint) could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression quarter#1425 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression loan_id#1318L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_year#1171L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression timestamp_month#1200L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU 22/11/29 12:41:46 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:46 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:47 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:47 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:47 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:48 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:48 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:48 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:48 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:49 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:49 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed [Stage 37:===================================================> (9 + 1) / 10] 22/11/29 12:41:50 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed 22/11/29 12:41:50 WARN GpuOverrides: *Exec will run on GPU *Partitioning will run on GPU *Exec will run on GPU *Expression cast(orig_channel#8609 as float) AS orig_channel#9231 will run on GPU *Expression cast(orig_channel#8609 as float) will run on GPU *Expression cast(first_home_buyer#8610 as float) AS first_home_buyer#9261 will run on GPU *Expression cast(first_home_buyer#8610 as float) will run on GPU *Expression cast(loan_purpose#8611 as float) AS loan_purpose#9291 will run on GPU *Expression cast(loan_purpose#8611 as float) will run on GPU *Expression cast(property_type#8612 as float) AS property_type#9321 will run on GPU *Expression cast(property_type#8612 as float) will run on GPU *Expression cast(occupancy_status#8613 as float) AS occupancy_status#9351 will run on GPU *Expression cast(occupancy_status#8613 as float) will run on GPU *Expression cast(property_state#8614 as float) AS property_state#9381 will run on GPU *Expression cast(property_state#8614 as float) will run on GPU *Expression cast(product_type#8615 as float) AS product_type#9411 will run on GPU *Expression cast(product_type#8615 as float) will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) AS relocation_mortgage_indicator#9441 will run on GPU *Expression cast(relocation_mortgage_indicator#8616 as float) will run on GPU *Expression cast(seller_name#8617 as float) AS seller_name#9471 will run on GPU *Expression cast(seller_name#8617 as float) will run on GPU *Expression cast(mod_flag#8618 as float) AS mod_flag#9501 will run on GPU *Expression cast(mod_flag#8618 as float) will run on GPU *Expression cast(orig_interest_rate#8619 as float) AS orig_interest_rate#9531 will run on GPU *Expression cast(orig_interest_rate#8619 as float) will run on GPU *Expression cast(orig_upb#8620 as float) AS orig_upb#9561 will run on GPU *Expression cast(orig_upb#8620 as float) will run on GPU *Expression cast(orig_loan_term#8621 as float) AS orig_loan_term#9591 will run on GPU *Expression cast(orig_loan_term#8621 as float) will run on GPU *Expression cast(orig_ltv#8622 as float) AS orig_ltv#9621 will run on GPU *Expression cast(orig_ltv#8622 as float) will run on GPU *Expression cast(orig_cltv#8623 as float) AS orig_cltv#9651 will run on GPU *Expression cast(orig_cltv#8623 as float) will run on GPU *Expression cast(num_borrowers#8624 as float) AS num_borrowers#9681 will run on GPU *Expression cast(num_borrowers#8624 as float) will run on GPU *Expression cast(dti#8625 as float) AS dti#9711 will run on GPU *Expression cast(dti#8625 as float) will run on GPU *Expression cast(borrower_credit_score#8626 as float) AS borrower_credit_score#9741 will run on GPU *Expression cast(borrower_credit_score#8626 as float) will run on GPU *Expression cast(num_units#8627 as float) AS num_units#9771 will run on GPU *Expression cast(num_units#8627 as float) will run on GPU *Expression cast(zip#8628 as float) AS zip#9801 will run on GPU *Expression cast(zip#8628 as float) will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) AS mortgage_insurance_percent#9831 will run on GPU *Expression cast(mortgage_insurance_percent#8629 as float) will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) AS current_loan_delinquency_status#9861 will run on GPU *Expression cast(current_loan_delinquency_status#8630 as float) will run on GPU *Expression cast(current_actual_upb#8631 as float) AS current_actual_upb#9891 will run on GPU *Expression cast(current_actual_upb#8631 as float) will run on GPU *Expression cast(interest_rate#8632 as float) AS interest_rate#9921 will run on GPU *Expression cast(interest_rate#8632 as float) will run on GPU *Expression cast(loan_age#8633 as float) AS loan_age#9951 will run on GPU *Expression cast(loan_age#8633 as float) will run on GPU *Expression cast(msa#8634 as float) AS msa#9981 will run on GPU *Expression cast(msa#8634 as float) will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) AS non_interest_bearing_upb#10011 will run on GPU *Expression cast(non_interest_bearing_upb#8635 as float) will run on GPU *Expression cast(delinquency_12#8636 as float) AS delinquency_12#10041 will run on GPU *Expression cast(delinquency_12#8636 as float) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression orig_channel#8609 ASC NULLS FIRST will run on GPU *Expression first_home_buyer#8610 ASC NULLS FIRST will run on GPU *Expression loan_purpose#8611 ASC NULLS FIRST will run on GPU *Expression property_type#8612 ASC NULLS FIRST will run on GPU *Expression occupancy_status#8613 ASC NULLS FIRST will run on GPU *Expression property_state#8614 ASC NULLS FIRST will run on GPU *Expression product_type#8615 ASC NULLS FIRST will run on GPU *Expression relocation_mortgage_indicator#8616 ASC NULLS FIRST will run on GPU *Expression seller_name#8617 ASC NULLS FIRST will run on GPU *Expression mod_flag#8618 ASC NULLS FIRST will run on GPU *Expression orig_interest_rate#8619 ASC NULLS FIRST will run on GPU *Expression orig_upb#8620 ASC NULLS FIRST will run on GPU *Expression orig_loan_term#8621 ASC NULLS FIRST will run on GPU *Expression orig_ltv#8622 ASC NULLS FIRST will run on GPU *Expression orig_cltv#8623 ASC NULLS FIRST will run on GPU *Expression num_borrowers#8624 ASC NULLS FIRST will run on GPU *Expression dti#8625 ASC NULLS FIRST will run on GPU *Expression borrower_credit_score#8626 ASC NULLS FIRST will run on GPU *Expression num_units#8627 ASC NULLS FIRST will run on GPU *Expression zip#8628 ASC NULLS FIRST will run on GPU *Expression mortgage_insurance_percent#8629 ASC NULLS FIRST will run on GPU *Expression current_loan_delinquency_status#8630 ASC NULLS FIRST will run on GPU *Expression current_actual_upb#8631 ASC NULLS FIRST will run on GPU *Expression interest_rate#8632 ASC NULLS FIRST will run on GPU *Expression loan_age#8633 ASC NULLS FIRST will run on GPU *Expression msa#8634 ASC NULLS FIRST will run on GPU *Expression non_interest_bearing_upb#8635 ASC NULLS FIRST will run on GPU *Expression delinquency_12#8636 ASC NULLS FIRST will run on GPU *Exec will run on GPU *Expression coalesce(orig_channel#2643, 0) AS orig_channel#8609 will run on GPU *Expression coalesce(orig_channel#2643, 0) will run on GPU *Expression coalesce(first_home_buyer#3289, 0) AS first_home_buyer#8610 will run on GPU *Expression coalesce(first_home_buyer#3289, 0) will run on GPU *Expression coalesce(loan_purpose#3935, 0) AS loan_purpose#8611 will run on GPU *Expression coalesce(loan_purpose#3935, 0) will run on GPU *Expression coalesce(property_type#4581, 0) AS property_type#8612 will run on GPU *Expression coalesce(property_type#4581, 0) will run on GPU *Expression coalesce(occupancy_status#5227, 0) AS occupancy_status#8613 will run on GPU *Expression coalesce(occupancy_status#5227, 0) will run on GPU *Expression coalesce(property_state#5873, 0) AS property_state#8614 will run on GPU *Expression coalesce(property_state#5873, 0) will run on GPU *Expression coalesce(product_type#6519, 0) AS product_type#8615 will run on GPU *Expression coalesce(product_type#6519, 0) will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) AS relocation_mortgage_indicator#8616 will run on GPU *Expression coalesce(relocation_mortgage_indicator#7165, 0) will run on GPU *Expression coalesce(seller_name#7811, 0) AS seller_name#8617 will run on GPU *Expression coalesce(seller_name#7811, 0) will run on GPU *Expression coalesce(id#2011, 0) AS mod_flag#8618 will run on GPU *Expression coalesce(id#2011, 0) will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) AS orig_interest_rate#8619 will run on GPU *Expression coalesce(nanvl(orig_interest_rate#1754, null), 0.0) will run on GPU *Expression nanvl(orig_interest_rate#1754, null) will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) AS orig_upb#8620 will run on GPU *Expression coalesce(nanvl(orig_upb#1756, null), 0.0) will run on GPU *Expression nanvl(orig_upb#1756, null) will run on GPU *Expression coalesce(orig_loan_term#1759, 0) AS orig_loan_term#8621 will run on GPU *Expression coalesce(orig_loan_term#1759, 0) will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) AS orig_ltv#8622 will run on GPU *Expression coalesce(nanvl(orig_ltv#1766, null), 0.0) will run on GPU *Expression nanvl(orig_ltv#1766, null) will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) AS orig_cltv#8623 will run on GPU *Expression coalesce(nanvl(orig_cltv#1767, null), 0.0) will run on GPU *Expression nanvl(orig_cltv#1767, null) will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) AS num_borrowers#8624 will run on GPU *Expression coalesce(nanvl(num_borrowers#1768, null), 0.0) will run on GPU *Expression nanvl(num_borrowers#1768, null) will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) AS dti#8625 will run on GPU *Expression coalesce(nanvl(dti#1769, null), 0.0) will run on GPU *Expression nanvl(dti#1769, null) will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) AS borrower_credit_score#8626 will run on GPU *Expression coalesce(nanvl(borrower_credit_score#1770, null), 0.0) will run on GPU *Expression nanvl(borrower_credit_score#1770, null) will run on GPU *Expression coalesce(num_units#1775, 0) AS num_units#8627 will run on GPU *Expression coalesce(num_units#1775, 0) will run on GPU *Expression coalesce(zip#1779, 0) AS zip#8628 will run on GPU *Expression coalesce(zip#1779, 0) will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) AS mortgage_insurance_percent#8629 will run on GPU *Expression coalesce(nanvl(mortgage_insurance_percent#1780, null), 0.0) will run on GPU *Expression nanvl(mortgage_insurance_percent#1780, null) will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) AS current_loan_delinquency_status#8630 will run on GPU *Expression coalesce(current_loan_delinquency_status#39, 0) will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) AS current_actual_upb#8631 will run on GPU *Expression coalesce(nanvl(current_actual_upb#11, null), 0.0) will run on GPU *Expression nanvl(current_actual_upb#11, null) will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) AS interest_rate#8632 will run on GPU *Expression coalesce(nanvl(interest_rate#8, null), 0.0) will run on GPU *Expression nanvl(interest_rate#8, null) will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) AS loan_age#8633 will run on GPU *Expression coalesce(nanvl(loan_age#15, null), 0.0) will run on GPU *Expression nanvl(loan_age#15, null) will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) AS msa#8634 will run on GPU *Expression coalesce(nanvl(msa#31, null), 0.0) will run on GPU *Expression nanvl(msa#31, null) will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) AS non_interest_bearing_upb#8635 will run on GPU *Expression coalesce(nanvl(non_interest_bearing_upb#62, null), 0.0) will run on GPU *Expression nanvl(non_interest_bearing_upb#62, null) will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END AS delinquency_12#8636 will run on GPU *Expression CASE WHEN (delinquency_12#1217 > 0) THEN 1 ELSE 0 END will run on GPU *Expression (delinquency_12#1217 > 0) will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS seller_name#7811 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS relocation_mortgage_indicator#7165 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS product_type#6519 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_state#5873 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS occupancy_status#5227 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS property_type#4581 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS loan_purpose#3935 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS first_home_buyer#3289 will run on GPU *Exec will run on GPU *Exec will run on GPU *Expression id#2011 AS orig_channel#2643 will run on GPU *Exec will run on GPU *Exec will run on GPU *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#216 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU #Exec could run on GPU but is going to be removed because replacing sortMergeJoin with shuffleHashJoin #Expression loan_id#1748L ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed #Expression quarter#1855 ASC NULLS FIRST could run on GPU but is going to be removed because parent plan is removed *Exec will run on GPU [Stage 45:===================================================> (8 + 1) / 9] 22/11/29 12:41:53 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) 22/11/29 12:41:53 WARN GpuOverrides: ! cannot run on GPU because not all expressions can be replaced; GPU does not currently support the operator class org.apache.spark.sql.execution.DeserializeToObjectExec ! createexternalrow(orig_channel#9231, first_home_buyer#9261, loan_purpose#9291, property_type#9321, occupancy_status#9351, property_state#9381, product_type#9411, relocation_mortgage_indicator#9441, seller_name#9471, mod_flag#9501, orig_interest_rate#9531, orig_upb#9561, orig_loan_term#9591, orig_ltv#9621, orig_cltv#9651, num_borrowers#9681, dti#9711, borrower_credit_score#9741, num_units#9771, zip#9801, mortgage_insurance_percent#9831, current_loan_delinquency_status#9861, current_actual_upb#9891, interest_rate#9921, ... 32 more fields) cannot run on GPU because GPU does not currently support the operator class org.apache.spark.sql.catalyst.expressions.objects.CreateExternalRow @Expression orig_channel#9231 could run on GPU @Expression first_home_buyer#9261 could run on GPU @Expression loan_purpose#9291 could run on GPU @Expression property_type#9321 could run on GPU @Expression occupancy_status#9351 could run on GPU @Expression property_state#9381 could run on GPU @Expression product_type#9411 could run on GPU @Expression relocation_mortgage_indicator#9441 could run on GPU @Expression seller_name#9471 could run on GPU @Expression mod_flag#9501 could run on GPU @Expression orig_interest_rate#9531 could run on GPU @Expression orig_upb#9561 could run on GPU @Expression orig_loan_term#9591 could run on GPU @Expression orig_ltv#9621 could run on GPU @Expression orig_cltv#9651 could run on GPU @Expression num_borrowers#9681 could run on GPU @Expression dti#9711 could run on GPU @Expression borrower_credit_score#9741 could run on GPU @Expression num_units#9771 could run on GPU @Expression zip#9801 could run on GPU @Expression mortgage_insurance_percent#9831 could run on GPU @Expression current_loan_delinquency_status#9861 could run on GPU @Expression current_actual_upb#9891 could run on GPU @Expression interest_rate#9921 could run on GPU @Expression loan_age#9951 could run on GPU @Expression msa#9981 could run on GPU @Expression non_interest_bearing_upb#10011 could run on GPU @Expression delinquency_12#10041 could run on GPU !Expression obj#10070 cannot run on GPU because expression AttributeReference obj#10070 produces an unsupported type ObjectType(interface org.apache.spark.sql.Row) [Stage 54:> (0 + 4) / 4] 22/11/29 12:41:59 WARN BlockManager: Block rdd_293_2 could not be removed as it was not found on disk or in memory 22/11/29 12:41:59 WARN BlockManager: Block rdd_293_3 could not be removed as it was not found on disk or in memory 22/11/29 12:41:59 WARN BlockManager: Block rdd_293_1 could not be removed as it was not found on disk or in memory 22/11/29 12:41:59 WARN BlockManager: Block rdd_293_0 could not be removed as it was not found on disk or in memory 22/11/29 12:41:59 ERROR Executor: Exception in task 3.0 in stage 54.0 (TID 3401) java.lang.NoClassDefFoundError: Could not initialize class ml.dmlc.xgboost4j.java.XGBoostJNI at ml.dmlc.xgboost4j.java.Rabit.shutdown(Rabit.java:83) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.buildDistributedBooster(GpuXGBoost.scala:327) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.$anonfun$trainOnGpuInternal$1(GpuXGBoost.scala:254) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:855) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:855) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:365) at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:378) at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1508) at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1435) at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1499) at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1322) at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:376) at org.apache.spark.rdd.RDD.iterator(RDD.scala:327) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 22/11/29 12:41:59 ERROR Executor: Exception in task 2.0 in stage 54.0 (TID 3400) java.lang.NoClassDefFoundError: Could not initialize class ml.dmlc.xgboost4j.java.XGBoostJNI at ml.dmlc.xgboost4j.java.Rabit.shutdown(Rabit.java:83) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.buildDistributedBooster(GpuXGBoost.scala:327) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.$anonfun$trainOnGpuInternal$1(GpuXGBoost.scala:254) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:855) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:855) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:365) at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:378) at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1508) at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1435) at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1499) at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1322) at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:376) at org.apache.spark.rdd.RDD.iterator(RDD.scala:327) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 22/11/29 12:41:59 ERROR Executor: Exception in task 1.0 in stage 54.0 (TID 3399) java.lang.NoClassDefFoundError: Could not initialize class ml.dmlc.xgboost4j.java.XGBoostJNI at ml.dmlc.xgboost4j.java.Rabit.shutdown(Rabit.java:83) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.buildDistributedBooster(GpuXGBoost.scala:327) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.$anonfun$trainOnGpuInternal$1(GpuXGBoost.scala:254) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:855) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:855) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:365) at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:378) at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1508) at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1435) at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1499) at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1322) at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:376) at org.apache.spark.rdd.RDD.iterator(RDD.scala:327) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 22/11/29 12:41:59 ERROR Executor: Exception in task 0.0 in stage 54.0 (TID 3398) java.lang.NoClassDefFoundError: Could not initialize class ml.dmlc.xgboost4j.java.XGBoostJNI at ml.dmlc.xgboost4j.java.Rabit.shutdown(Rabit.java:83) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.buildDistributedBooster(GpuXGBoost.scala:327) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.$anonfun$trainOnGpuInternal$1(GpuXGBoost.scala:254) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:855) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:855) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:365) at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:378) at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1508) at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1435) at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1499) at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1322) at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:376) at org.apache.spark.rdd.RDD.iterator(RDD.scala:327) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 22/11/29 12:41:59 WARN TaskSetManager: Lost task 3.0 in stage 54.0 (TID 3401) (xxxxxxxxx.local executor driver): java.lang.NoClassDefFoundError: Could not initialize class ml.dmlc.xgboost4j.java.XGBoostJNI at ml.dmlc.xgboost4j.java.Rabit.shutdown(Rabit.java:83) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.buildDistributedBooster(GpuXGBoost.scala:327) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.$anonfun$trainOnGpuInternal$1(GpuXGBoost.scala:254) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:855) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:855) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:365) at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:378) at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1508) at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1435) at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1499) at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1322) at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:376) at org.apache.spark.rdd.RDD.iterator(RDD.scala:327) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 22/11/29 12:41:59 ERROR TaskSetManager: Task 3 in stage 54.0 failed 1 times; aborting job 22/11/29 12:41:59 ERROR RabitTracker: Uncaught exception thrown by worker: org.apache.spark.SparkException: Job aborted due to stage failure: Task 3 in stage 54.0 failed 1 times, most recent failure: Lost task 3.0 in stage 54.0 (TID 3401) (xxxxxx.local executor driver): java.lang.NoClassDefFoundError: Could not initialize class ml.dmlc.xgboost4j.java.XGBoostJNI at ml.dmlc.xgboost4j.java.Rabit.shutdown(Rabit.java:83) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.buildDistributedBooster(GpuXGBoost.scala:327) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.$anonfun$trainOnGpuInternal$1(GpuXGBoost.scala:254) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:855) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:855) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:365) at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:378) at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1508) at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1435) at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1499) at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1322) at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:376) at org.apache.spark.rdd.RDD.iterator(RDD.scala:327) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Driver stacktrace: at org.apache.spark.scheduler.DAGScheduler.failJobAndIndependentStages(DAGScheduler.scala:2672) at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2(DAGScheduler.scala:2608) at org.apache.spark.scheduler.DAGScheduler.$anonfun$abortStage$2$adapted(DAGScheduler.scala:2607) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49) at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:2607) at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1(DAGScheduler.scala:1182) at org.apache.spark.scheduler.DAGScheduler.$anonfun$handleTaskSetFailed$1$adapted(DAGScheduler.scala:1182) at scala.Option.foreach(Option.scala:407) at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:1182) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:2860) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2802) at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:2791) at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:49) at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:952) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2228) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2249) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2268) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2293) at org.apache.spark.rdd.RDD.$anonfun$foreachPartition$1(RDD.scala:1011) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112) at org.apache.spark.rdd.RDD.withScope(RDD.scala:406) at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:1009) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$$anon$1.run(GpuXGBoost.scala:190) Caused by: java.lang.NoClassDefFoundError: Could not initialize class ml.dmlc.xgboost4j.java.XGBoostJNI at ml.dmlc.xgboost4j.java.Rabit.shutdown(Rabit.java:83) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.buildDistributedBooster(GpuXGBoost.scala:327) at ml.dmlc.xgboost4j.scala.spark.rapids.GpuXGBoost$.$anonfun$trainOnGpuInternal$1(GpuXGBoost.scala:254) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2(RDD.scala:855) at org.apache.spark.rdd.RDD.$anonfun$mapPartitions$2$adapted(RDD.scala:855) at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:365) at org.apache.spark.rdd.RDD.$anonfun$getOrCompute$1(RDD.scala:378) at org.apache.spark.storage.BlockManager.$anonfun$doPutIterator$1(BlockManager.scala:1508) at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$doPut(BlockManager.scala:1435) at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1499) at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:1322) at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:376) at org.apache.spark.rdd.RDD.iterator(RDD.scala:327) at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) at org.apache.spark.scheduler.Task.run(Task.scala:136) at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548) at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504) at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750)