# Restaurant campaign randomized test templates These blank templates accompany the article **How to Measure Whether a Restaurant Email Campaign Actually Brought Guests Back**. They contain no actual guest records, example results or benchmark targets. They do not send email, randomize an audience, connect to a POS, or infer missing outcomes. ## Files and purpose - **campaign-randomized-test-template.csv** is the per-guest outcome log. It contains only column headers; add one row per originally assigned eligible profile - **campaign-test-summary-template.csv** is an optional aggregate reporting sheet. Its Email and Holdout rows have empty counts, deliberately not zeros - **guest-wifi-diagnostic-worksheet.html** is a separate two-page printable staff and network-owner troubleshooting worksheet ## Decide the experiment before assigning anyone Record these choices in the campaign brief before the send: 1. Experiment ID, owner, participating venue or locations, and reporting currency 2. Exact eligibility and exclusion rules, with permission to send marketing already established 3. One frozen set of eligible guest profiles, deduplicated before random assignment 4. Randomization method, allocation ratio and assignment date. Keep a reproducible assignment record; do not reassign people after seeing results 5. Common index timestamp and time zone. The outcome window is **[index timestamp, index timestamp plus 14 days)**, including the first instant and excluding the ending instant. Give Holdout the same scheduled index time as Email 6. Primary outcome: at least one observed, matched qualifying return during the window. Predefine which locations count, how sessions are deduplicated, and whether a later session on the same service day counts as a new visit 7. Observation source, identity-matching method, expected completeness, and what to do about outages or unresolved matches 8. Other campaign restrictions, a record of contamination, and a stop rule for operational or customer-safety problems Use the same guest-profile unit for eligibility, assignment and reporting. Do not use sessions in the denominator and people in the numerator. Do not merge different devices into a person without a justified identity match. A WiFi-observed return is not a complete count of every physical return to the venue. ## Per-guest field dictionary ### guest_key A unique pseudonymous identifier for the originally assigned eligible profile. Do not put an email address, phone number, raw MAC address or name in this file. Keep any lookup separately in the venue's authorized system. Pseudonymous data can still be personal data and needs appropriate access controls and retention. ### assigned_arm Exactly **Email** or **Holdout**, based on the original random assignment. Keep this assignment when an email bounces, is not opened, or is not delivered. Do not move non-openers into Holdout. The template does not perform random assignment. ### index_time The prespecified campaign timestamp in ISO 8601 with a UTC offset, such as the format YYYY-MM-DDTHH:MM:SS+00:00. Use the same scheduled timestamp for both groups in a single batch. If the study has genuine separate batches, record and analyze those consistently. Do not use a guest's email-open time as the start. ### returned_14d Use **1** if at least one qualifying matched return is observed during the fixed window. Use **0** only when the window is complete, observation is reliable for the defined outcome, and no qualifying return was observed. Leave blank when unresolved identity or an outage means the outcome is unknown. A reliable zero for WiFi-observed activity does not prove the guest did not visit physically. ### net_contribution_14d Optional sum of reliable matched transaction contribution during the same window, in the brief's single currency. Define the cost components before the test and use the same definition for both arms. Leave blank when the required transaction data or reliable join is unavailable. Do not substitute total sales, average checks, email opens, coupon redemption counts or an assumed margin for observed contribution. Avoid counting discount costs twice if they are already included in contribution. ### observation_status Use one of **complete**, **unresolved_identity**, or **outage**. If more than one issue applies, choose the principal reason consistently and document it in the experiment notes. Do not change a missing outcome to zero to make the sheet complete. Assess contribution coverage separately if it differs from return-visit coverage. ## Aggregate field dictionary - **experiment_id:** the brief's ID, identical in both rows - **assigned_arm:** Email or Holdout - **assigned_eligible_n:** the original number of eligible profiles randomized to that arm, including bounces and non-openers - **complete_observation_n:** profiles with observation_status = complete - **unresolved_identity_n:** profiles with observation_status = unresolved_identity - **outage_n:** profiles with observation_status = outage - **observed_returners_14d_n:** profiles with a known returned_14d value of 1; count each profile once - **matched_net_contribution_14d:** optional sum of reliable contribution values; do not present it as a complete-arm total if values are missing - **contribution_coverage_status:** explain whether contribution data cover the full assigned arm or are incomplete; leave blank if contribution was not measured - **notes:** redacted operational context, observation problems, contamination, or analysis restrictions; no guest identifiers ## Calculations and interpretation When primary-outcome observation is complete for everyone: - Email return rate = Email returners / Email originally assigned profiles - Holdout return rate = Holdout returners / Holdout originally assigned profiles - Absolute lift in percentage points = 100 × (Email rate − Holdout rate) - Estimated additional observed returners within the Email arm = (Email rate − Holdout rate) × Email assigned profiles These are intention-to-treat comparisons using original assignment. Report both arm sizes, outcome counts, rates, the window and an appropriate uncertainty interval. A positive point estimate alone does not establish that the campaign worked. Do not stop early because one interim result looks favorable. Small samples and sparse outcomes need particular statistical care. If outcomes are missing, disclose counts by arm. A complete-case rate may be reported descriptively but is not automatically an unbiased intention-to-treat estimate. Simple logical sensitivity bounds for each arm are: - Lower possible rate = known returners / all assigned profiles - Upper possible rate = (known returners + profiles with unknown outcomes) / all assigned profiles These are missing-outcome bounds, not confidence intervals. They assume every unknown outcome could be either 0 or 1. Seek an appropriate analysis plan rather than silently imputing zeros or discarding profiles differently across arms. Only with reliable contribution measurement across the assigned groups may the difference in average contribution be estimated using the original assigned denominators. Separately account for prespecified incremental campaign costs, consistent with the contribution definition. WiFi records alone do not establish matched transaction revenue or profit. Coupon redemptions show association with a campaign; they do not by themselves measure visits caused by it. ## Checks before reporting - Each guest_key appears once and has one original assigned_arm - No result was classified using whether the guest opened or clicked the email - Both arms share the same outcome definition, measurement rules and comparable window - Complete plus unresolved-identity plus outage counts equal the assigned count in each arm - Known returners do not exceed the assigned count; 0/1 values and observation status agree - Any missing outcomes or unmatched transactions are visible, not converted to zero - Contribution uses one currency and a documented cost basis - Findings are described as WiFi-observed returns when WiFi is the observation source - Any email follow-up respects consent and unsubscribe preferences in both groups ## Sources and limitations - [Microsoft Research on controlled experiments](https://www.microsoft.com/en-us/research/group/experimentation-platform-exp/) provides the wider experimental-method context; this worksheet is an operational aid, not a substitute for statistical design - [Mailchimp import guidance](https://mailchimp.com/help/import-contacts-mailchimp/) explains marketing subscription status and the need to have permission before importing subscribed contacts - [Apple private WiFi addresses](https://support.apple.com/en-gb/102509) explains why device identifiers can change and should not automatically be treated as stable people - [VoqadoWiFi's existing attribution guide](https://www.voqadowifi.com/blog/wifi-campaign-attribution-guide) provides product-category context. This template deliberately distinguishes campaign-associated redemptions from causal lift The accompanying article contains a clearly labeled hypothetical worked example. No hypothetical rates or results are prefilled into these CSVs. New article drafts remain subject to review; this template does not assume a native Voqado export or POS integration exists.