In a benchmark run by the Azure Foundry team, encrypted Blackbox delivers 3.02× Azure’s mean throughput and 75.1% lower mean query latency, with close quality scores.
The Azure Foundry team ran this benchmark of DeepSeek v4.1 flash (xhigh), comparing Azure with regular and encrypted inference on BLACKBOX AI. In the reported runs, Blackbox’s encrypted path delivered 245.88 tokens/s mean generation throughput and 30.43 seconds mean query latency, versus 81.47 tokens/s and 122.15 seconds on Azure.
3.02×
mean throughput vs. Azure
75.1%
lower mean query latency
0.379 s
encrypted mean TTFT
The performance figures compare DeepSeek v4.1 flash (xhigh) across Azure, regular Blackbox, and encrypted Blackbox. Orange consistently identifies the encrypted path. The results cover 230 requests on Azure, 179 on regular Blackbox, and 330 on encrypted Blackbox.
Three times the throughput
Regular Blackbox averaged 283.74 tokens/s. Encrypted Blackbox averaged 245.88 tokens/s, compared with 81.47 tokens/s on Azure: 3.48× and 3.02× Azure’s mean throughput, respectively.
Source: Azure Foundry team benchmark summary. Azure n=230; Blackbox regular n=179; encrypted n=330. All panels share the same zero-based scale.
Encrypted throughput was 13.3% below regular Blackbox. The comparison with Azure still shows a threefold increase in mean generation throughput.
TTFT at 0.379 seconds
Mean time to first token was 0.379 seconds on encrypted Blackbox, 0.399 seconds on regular Blackbox, and 10.71 seconds on Azure. The panels below compare the encrypted path with Azure at the mean, median, and P99.
Source: Azure Foundry team benchmark summary. Azure n=230; Blackbox regular n=179; encrypted n=330. All panels share the same zero-based scale.
At P99, Azure’s TTFT was 51.385 seconds, versus 0.440 seconds on encrypted Blackbox. The bars use a common linear scale; the exact labels keep the sub-second values readable. All five reported statistics are retained in the full results below.
Shorter queries, including the tail
Mean query latency was 122.15 seconds on Azure, 26.82 seconds on regular Blackbox, and 30.43 seconds on encrypted Blackbox. The encrypted mean was 75.1% below Azure’s and 13.5% above regular Blackbox’s.
Source: Azure Foundry team benchmark summary. Azure n=230; Blackbox regular n=179; encrypted n=330. All panels share the same zero-based scale.
At P99, encrypted Blackbox completed queries in 62.33 seconds, compared with 414.70 seconds on Azure. Percentiles summarize separate runs; they do not pair individual requests.
Blackbox outperforms on throughput, speed and security with no quality loss
The speed gains come with close reported quality scores on the three shared accuracy datasets. Encrypted Blackbox is 2.4 percentage points above Azure on MuSR, 0.8 points below on tau2-telecom, and 0.2 points below on ChemBench. These results support comparable observed quality, rather than a claim that every score is unchanged.
Figure 4. Quality remains close across shared datasets
DeepSeek v4.1 flash (xhigh)
Higher is better
Figure 4. Quality remains close across shared datasets
Source: supplied quality summaries. Azure uses Correct/Dataset. All three panels use a 0–100% scale; FrontierScience uses a different metric and is shown separately.
Each dataset has its own panel, with the same serving-path order and 0–100% scale. Against regular Blackbox, encrypted accuracy differs by +0.8 points on MuSR, −0.8 on tau2-telecom, and −0.3 on ChemBench.
FrontierScience: partial-credit scores
Blackbox reports a mean score including partial credit of 59.1% for regular inference and 57.7% for encrypted inference, with 74 and 71 fully correct answers out of 160 examples.
Figure 5. FrontierScience scores differ by 1.4 points
DeepSeek v4.1 flash (xhigh)
Higher is better
Figure 5. FrontierScience scores differ by 1.4 points
Source: supplied Blackbox quality summary. 160 examples per path. Mean score includes partial credit; fully correct answers: regular 74, encrypted 71.
Azure reports 90 correct, 155 judged, and 160 dataset rows, alongside 57.7% Correct/Judged and 55.9% Correct/Dataset. Those percentages do not match simple division of the displayed counts (90/155 ≈ 58.1%; 90/160 ≈ 56.3%), and the summary does not establish equivalence with Blackbox’s partial-credit metric. Azure is therefore excluded from this figure.
Evaluation conditions
Encrypted Blackbox recorded zero errors across 3,309 examples; regular Blackbox recorded one on FrontierScience. Regular parallelism was 4 throughout. Encrypted parallelism was 1 on MuSR, 4 on tau2-telecom, 5 on FrontierScience, and 10 on ChemBench. Dataset names alone do not establish identical evaluation settings.
The cost of encryption in these runs
Encrypted Blackbox’s mean throughput was 13.3% lower than regular Blackbox’s, and mean query latency was 13.5% higher. Mean TTFT was 5.0% lower. These are observed differences between runs, not an isolated measurement of encryption overhead: request counts differ, and the summary does not document matched prompts, concurrency, hardware, cache state, or repeat trials.
Mean emitted output length was 34,522 characters on Azure, 34,435 on regular Blackbox, and 33,846 on encrypted Blackbox. Similar character counts provide length context; they do not establish equal token counts or equivalent answers.
Full performance results and calculation notes
The following tables reproduce the supplied performance summary for DeepSeek v4.1 flash (xhigh).
Serving path
Requests
Azure
230
Blackbox regular
179
Blackbox encrypted
330
TTFT (s)
Statistic
Azure
Blackbox regular
Blackbox encrypted
Encryption cost
Mean
10.71
0.399
0.379
-5.0%
P50
7.437
0.395
0.378
-4.3%
P90
22.902
0.458
0.404
-11.8%
P95
30.197
0.482
0.416
-13.7%
P99
51.385
0.544
0.44
-19.1%
Query latency (s)
Statistic
Azure
Blackbox regular
Blackbox encrypted
Encryption cost
Mean
122.15
26.82
30.43
13.5%
P50
94.87
25.49
30.94
21.4%
P90
254.69
44.02
49.3
12.0%
P95
290.03
49.97
54.28
8.6%
P99
414.7
55.99
62.33
11.3%
Generation throughput (tokens/s)
Statistic
Azure
Blackbox regular
Blackbox encrypted
Encryption cost
Mean
81.47
283.74
245.88
13.3%
P50
72.48
278.18
242.6
12.8%
P90
135.4
319.75
277.21
13.3%
P95
163.7
346.12
302.61
12.6%
P99
219.21
378.94
325.65
14.1%
Total emitted characters (characters/response)
Statistic
Azure
Blackbox regular
Blackbox encrypted
Encryption cost
Mean
34,522
34,435
33,846
-1.7%
P50
33,995
34,695
35,682
2.8%
P90
56,046
55,043
52,794
-4.1%
P95
60,030
59,223
58,458
-1.3%
P99
72,604
69,340
66,169
-4.6%
The source calls its final column “Encryption cost.” Latency and emitted-character changes use 100 × (encrypted / regular − 1). Throughput uses 100 × (1 − encrypted / regular), so a positive number represents a throughput decrease. Percentages are reproduced as reported.