PlatformGuard

Experiments

Test tone and pacing changes on real users, with a proper significance test.

Define an experiment with two variants, each a set of engine settings, and Humane assigns users and applies the right overlay on every call. Results compare the variants with lift and a two-proportion z-test.

Experiment · warmer_check_ins

Variant Acontrol

empathy_weighting: 70

7-day return

38%

Variant Bwarmer

empathy_weighting: 85

7-day return

41%

z-test · p = 0.21Not significant
Illustration of an A/B experiment (example data)
How it works

Find out whether warmer is better, or just slower

Experiments let you change how the AI behaves for half your users without branching your code. Each response says which variant the user is in, and the experiment's start and end fire webhooks.

  • Variants A and B, each a config overlay such as empathy or pacing
  • The assignment comes back on every response under experiments
  • Comparison with lift and a two-proportion z-test p-value
  • experiment.started and experiment.concluded events

Behavior Score · last 7 days

74/ 100 · composite
  • retention_deltaweight 30%
  • conversation_depthweight 25%
  • proactive_response_rateweight 20%
  • abandonment_inverseweight 15%
  • sentiment_trendweight 10%
Illustration of the Behavior Score (example data)
On the response

Know which variant a user saw

python
r = client.process(user_id="user_42", message=text)

for e in r.experiments:
    print(e.experiment_name, e.variant, e.config_overlay)
# warmer_check_ins b {'empathy_weighting': 85}

Experiment questions

Something else? Ask us.

Every AI should know who it's talking to.

The safety and human-oversight layer for emotional AI: user state across sessions, risk alerts, safety gates and human takeover, with every decision explained.