nifectl abtest Command Reference
Manage A/B tests on an application.
Overview​
The abtest commands manage an A/B test: two versions of an app (variant A and variant B) run side by side and traffic is split between them. They mirror the A/B Testing tab in the nife.io dashboard.
Deploy the app with --deployment_strategy ab-testing first. Then use these commands to watch the results, change the traffic split and finish the test.
Subcommands​
| Subcommand | Description |
|---|---|
abtest status | Show the current A/B test status |
abtest results | Show the A/B test results |
abtest traffic | Change the traffic split between variant A and variant B |
abtest configure | Set the test name, duration, success metrics and starting split |
abtest declare-winner | Send all traffic to the winning variant and end the test |
abtest end | End the A/B test without declaring a winner |
nifectl abtest <subcommand> [flags]
Options​
-a, --app string App name to operate on (env: NIFE_APP)
-c, --config string Path to an app config file or directory containing one (env: NIFE_APP_CONFIG)
-h, --help Help for abtest
Global Options​
-t, --access-token string Nife API Access Token
-j, --json JSON output format
-V, --verbose Verbose / debug output
abtest status​
Show the current A/B test status.
Display whether an A/B test is running, the version, health and pod count of variant A and variant B, and the current traffic split, mirroring the A/B Testing tab in the nife.io dashboard.
nifectl abtest status [flags]
Options:
-a, --app string App name to operate on (env: NIFE_APP)
-c, --config string Path to an app config file or directory containing one (env: NIFE_APP_CONFIG)
abtest results​
Show the A/B test results.
Compare variant A and variant B: users, conversions, conversion rate, average latency and error rate, plus the leading variant and how statistically significant the difference is.
Use --watch to keep refreshing until you press Ctrl+C.
nifectl abtest results [flags]
Options:
-a, --app string App name to operate on (env: NIFE_APP)
-c, --config string Path to an app config file or directory containing one (env: NIFE_APP_CONFIG)
--watch Keep refreshing the results until you press Ctrl+C
--interval int Seconds between refreshes when using --watch (default 10)
abtest traffic​
Change the traffic split between variant A and variant B.
Set the percentage of traffic sent to each variant. Give --traffic-a or --traffic-b (the other one gets the rest). Prompts if you give neither.
nifectl abtest traffic [flags]
Options:
-a, --app string App name to operate on (env: NIFE_APP)
-c, --config string Path to an app config file or directory containing one (env: NIFE_APP_CONFIG)
--traffic-a int Percentage of traffic sent to variant A (0-100). Variant B gets the rest.
--traffic-b int Percentage of traffic sent to variant B (0-100). Variant A gets the rest.
Examples:
nifectl abtest traffic -a myapp --traffic-a 70
nifectl abtest traffic -a myapp --traffic-a 70 --traffic-b 30
abtest configure​
Set the test name, duration, success metrics and starting split.
Save the settings of an A/B test. The app must already be deployed with --deployment_strategy ab-testing. Give at least one flag.
nifectl abtest configure [flags]
Options:
-a, --app string App name to operate on (env: NIFE_APP)
-c, --config string Path to an app config file or directory containing one (env: NIFE_APP_CONFIG)
--test-name string Display name for the test
--duration-days int How many days the test should run
--success-metric strings A metric that decides success (repeatable), e.g. conversion
--traffic-a int Starting percentage of traffic sent to variant A (0-100). Variant B gets the rest.
--traffic-b int Starting percentage of traffic sent to variant B (0-100). Variant A gets the rest.
Examples:
nifectl abtest configure -a myapp --test-name "New checkout" --duration-days 14 \
--success-metric conversion --success-metric latency --traffic-a 80
abtest declare-winner​
Send all traffic to the winning variant and end the test.
Route 100% of traffic to variant A or variant B and end the test. The losing variant is deleted unless you pass --keep-loser.
Like the dashboard, this is blocked until the results are statistically significant (0.95). Use --force to skip that check.
nifectl abtest declare-winner <a|b> [flags]
Options:
-a, --app string App name to operate on (env: NIFE_APP)
-c, --config string Path to an app config file or directory containing one (env: NIFE_APP_CONFIG)
--keep-loser Keep the losing variant running instead of deleting it
--force Declare the winner even if the results are not statistically significant yet
-y, --yes Accept all confirmations
Examples:
nifectl abtest declare-winner b -a myapp
abtest end​
End the A/B test without declaring a winner.
End the test. Traffic stays at the current split and both variants keep running. Use declare-winner instead to pick a variant.
nifectl abtest end [flags]
Options:
-a, --app string App name to operate on (env: NIFE_APP)
-c, --config string Path to an app config file or directory containing one (env: NIFE_APP_CONFIG)
-y, --yes Accept all confirmations
Best Practices​
- Run
nifectl abtest results --watchto follow the numbers live; pressCtrl+Cto stop. declare-winneris blocked until the result is statistically significant (0.95), just like the dashboard. Use--forceonly when you have decided without it.- The losing variant is deleted when you declare a winner. Add
--keep-loserto keep it. - Use
abtest endto stop the test without choosing a winner — both variants keep running at the current split.
See Also​
nifectl canary– Roll out to a share of pods firstnifectl bg– Blue-green deploymentsnifectl shadow– Try a new version without serving usersnifectl redeploy– Redeploy an applicationnifectl– The Nife CLI home command
Summary​
Nifectl Abtest brings the whole A/B testing flow — configure, compare, shift traffic and pick a winner — to the command line.