Propensity Scoring Pipeline

BigQuery ML + XGBoost

The propensity scoring pipeline is the core of ConversionOS. It takes raw customer/prospect data, engineers predictive features, trains gradient-boosted models, and outputs calibrated probability scores.

Pipeline Architecture

Raw Events → Feature Engineering → Training → Scoring → Tier Assignment → CDP Export
                  (BigQuery)       (BQML)    (BQML)     (SQL Logic)     (Scheduled)

Feature Categories

Behavioral Features (25-40 features)

Feature	Description	Signal
pageviews_7d	Page views in last 7 days	Recency
engagement_recency_ratio	7d events / 30d events	Momentum
product_page_views	Views of product/pricing pages	Intent
form_starts	Form interactions without completion	Friction signal
return_visit_rate	Sessions with prior visits / total	Interest depth

Transactional Features (10-15 features)

Feature	Description	Signal
days_since_last_purchase	Recency of last transaction	Engagement decay
purchase_frequency_90d	Transactions per 90 days	Loyalty
avg_order_value	Average transaction amount	Value tier
product_diversity	Unique categories purchased	Breadth

Engagement Features (15-20 features)

Feature	Description	Signal
email_open_rate_30d	Opens / sends in 30 days	Channel engagement
email_click_decay	7d clicks / 30d clicks	Email momentum
call_center_contacts	Support interactions	Risk/friction
nps_score	Net Promoter Score (if available)	Satisfaction

External Features (5-10 features)

Feature	Description	Signal
competitive_density	# competitors in service area	Market pressure
credit_tier	Appended from identity resolution	Qualification
household_income_band	Demographic append	Capacity

Model Configuration

CREATE OR REPLACE MODEL `project.conversionos.acquisition_propensity`
OPTIONS(
  model_type = 'BOOSTED_TREE_CLASSIFIER',
  input_label_cols = ['converted'],
  -- Tree parameters
  num_parallel_tree = 5,
  max_tree_depth = 6,
  learn_rate = 0.1,
  subsample = 0.8,
  -- Regularization
  l1_reg = 1.0,
  l2_reg = 1.0,
  min_split_loss = 0.1,
  -- Training config
  early_stop = TRUE,
  min_rel_progress = 0.001,
  max_iterations = 100,
  data_split_method = 'RANDOM',
  data_split_eval_fraction = 0.2,
  -- Explainability
  enable_global_explain = TRUE
) AS
SELECT * EXCEPT(user_id) FROM `project.conversionos.feature_table`;

Score Calibration

Raw model probabilities must be calibrated before use in business rules:

1. Platt scaling — Apply logistic regression on holdout set to calibrate probabilities
2. Threshold tuning — Set tier boundaries based on precision/recall tradeoff at each tier
3. Business validation — Verify that HIGH tier converts at 3-5x the base rate

Monitoring

Schedule weekly checks for:

• Score distribution shift — KL divergence between current and training distributions
• Feature importance drift — Top 10 features should remain stable
• Conversion rate by tier — Each tier should maintain expected conversion multiples
• AUC-ROC trend — Should not degrade more than 0.02 between retraining cycles