Privacy Budget

Overview

As mentioned in the previous section, epsilon ($ϵ$) and delta ($δ$) are the base parameters that quantify and manage privacy protection in differential privacy. AGENT distributes its values to teams and data scientists through the Privacy Budget. All Datasets have a lifetime Privacy Budget, which is then allocated to Teams and data scientists. Each query they make spends Epsilon and Delta from the Privacy Budget. The amount of Epsilon and Delta spent on their analysis also determines the tradeoff between safeguarding the privacy of individual data points and preserving the dataset's utility for data scientists.

Administrators can ensure privacy effectively by creating a hierarchy and allocating Privacy Budget amounts, even when different teams and members use the same dataset.

Budget Accountant

AGENT maintains an in‑built differential‑privacy accountant that tracks the cumulative privacy loss $(\varepsilon,\delta)$ over the lifetime of a project or team. Given a list of spends (one record per query to AGENT), the accountant chooses the tightest valid composition theorem and returns the total spent.

Basic Composition (always valid)

If every query $i$ provides $(\varepsilon_i,\delta_i)$‑DP, simply add

$$\varepsilon_{\text{total}}=\sum_{i=1}^{n}\varepsilon_i, \qquad \delta_{\text{total}}=\sum_{i=1}^{n}\delta_i.$$

This is the worst‑case bound but applies to all mechanisms.

Advanced Composition (tighter for heterogeneous $\varepsilon$/$\delta$)

Fix an extra slack parameter $\delta' > 0$. For $(\varepsilon_i,\delta_i)$‑DP queries you may use

$$\varepsilon_{\text{total}} = \sqrt{\,2\log\!\bigl(1/\delta'\bigr)\,\sum_{i=1}^{n}\varepsilon_i^{2}} + \sum_{i=1}^{n}\varepsilon_i\bigl(e^{\varepsilon_i}-1\bigr),$$ $$\delta_{\text{total}} = \sum_{i=1}^{n}\delta_i+\delta'.$$

When every $\varepsilon_i \ll 1$, the second term above is often negligible.

zCDP Composition (when rho=True)

For mechanisms that satisfy zero‑Concentrated DP (zCDP) with parameters $\rho_i$ (e.g. Gaussian mechanism):

$$\rho_{\text{total}}=\sum_{i=1}^{n}\rho_i.$$

Convert back to $(\varepsilon,\delta)$ with

$$\varepsilon(\delta)=\rho_{\text{total}}+\sqrt{\,4\rho_{\text{total}}\log\!\bigl(1/\delta\bigr)}.$$

If you only know $(\varepsilon_i,\delta_i)$ for each query you can lift them to zCDP via

$$\rho_i=\tfrac12\,\varepsilon_i^{2}+\varepsilon_i\sqrt{2\log\!\bigl(1/\delta_i\bigr)}.$$

Understanding δ′ (delta‑prime) in Advanced Composition

In the advanced–composition theorem we introduce an extra parameter δ′ > 0. It is not one of the per‑query δ’s; instead, it is a tunable slack variable that lets the accountant trade a little extra failure probability for a tighter bound on ε. AGENT uses the remaining delta in the budget for this slack variable. For example, if your total delta budget is ($\delta_\text{total}$) and you have already spent ($\delta_1$) on queries, then the remaining delta budget is ($\delta_{\text{total}} - \delta_1$). This value will be used as the value of δ′ in the advanced composition theorem.

Warning

When a lot of $\delta$ is remaining the budget, you might see the total $\epsilon$ being very low since we get a tight bound on $\epsilon$ using the advanced composition theorem. This is because the slack variable δ′ is used to get a tighter bound on $\epsilon$. When you start exhausting the delta budget, you will see the total epsilon increasing. This is because the slack variable δ′ reduces as you exhaust δ from your budget. When the delta budget is completely exhausted, the accountant uses a very small value $10^{-12}$ as the value of δ′ and the same is added to your total spent. Hence, you might see that that total $\delta$ that you spent is slightly more than your budget when you exhaust your delta completely.

Total spent calculation

For low epsilon values, the accountant uses advanced composition. High values of epsilon (Usually ($ϵ > 1.0$), unless set otherwise by admin) are handled with basic composition because advanced composition performs worse than basic. The accountant also uses the zCDP composition when gaussian noise mechanism is used in the query.

Privacy Budget Hierarchy

The Privacy Budget is handled hierarchically, and different users with different roles control its distribution. The process begins when a user creates a dataset, becomes the Dataset Admin, and defines the total Privacy Budget available. The Dataset Admin manages and delegates the Privacy Budget amount each team and member can use.

The primary Privacy Budget hierarchy flow works as follows:

1

Dataset Admins determine the dataset's lifetime Privacy Budget.

2

Dataset Admins allocate parts of this Privacy Budget to the teams working on this dataset.

3

Team Admins can then allocate Privacy Budget amounts to each Team Member.

4

Members can spend the available Privacy Budget on data queries.

Team Admins and Members can request more Privacy Budget:

1

Team Admins can request more Privacy Budget for the Dataset Admin.

2

Members can request more Privacy Budget for the Team Admin if they are part of a team or directly go to the Dataset Admin.

Privacy Budget Hierarchy Example

The diagram below presents an example of Privacy Budget distribution.

In the diagram, we can observe the following:

Dataset Budget

The dataset has a lifetime Privacy Budget of 5 and the Dataset Admin allocated the whole Privacy Budget to two teams.

• Team A received a Privacy Budget of 2
• Team B received a Privacy Budget of 3.

Team Privacy Budget

Each team has two Members.

• Team A: The Team Admin for this team split the Privacy Budget equally, so each member received a Privacy Budget of 1.
• Team B: In this case, the Team admin allocated a Privacy Budget of 2 to Member C and a Privacy Budget of 1 to Member D

Member Privacy Budget

Each Member can now spend the allocated as needed. At any moment, the Member can request an additional Privacy Budget from its Team Admin.

In this example, both the Dataset and Team Admins allocated their whole Privacy Budget, but they can also allocate it partially, leaving available Privacy Budget that can be allocated later.

Spending Privacy Budget for Your Needs

When spending the Privacy Budget to allocate Epsilon ($\epsilon$) and Delta ($\delta$) to teams and members, consider the following:

1

Understand the parameters

It is essential to understand how Epsilon and Delta affect the utility or accuracy of the data analysis.

2

Consider the context of the data usage

The nature of the data and its intended use are crucial in determining $\epsilon$ and $\delta$. Highly sensitive data like health records may require a smaller $\epsilon$ for stronger privacy.

3

Define a desired level of privacy

Determine the acceptable level of privacy risk. In scenarios where individual privacy is critical, opting for a smaller $\epsilon$ is advisable.

4

Understand the data analysis goals

Consider the required accuracy and specificity of the data analysis results. For broader, less granular insights, a smaller ε may suffice.

Selecting suitable Epsilon ($\epsilon$) and delta ($\delta$) values is a critical decision that impacts the balance between data utility and privacy. Depending on the scenario, consider the following factors:

Data sensitivity

Highly sensitive data needs lower $\epsilon$ values to ensure data is more strictly protected.

Population size

Larger datasets typically will have better signal-to-noise ratios, mitigating the effect of the noise on the insights created.

Data access frequency

Frequent access to data for analysis might require more restrictive $\epsilon$ settings to maintain privacy over time, especially if data is reused in a new analysis, and thus the Privacy Budgets can compound.

Collaborative environments

When multiple parties are involved, consider the cumulative privacy risk and adjust $\epsilon$ and $\delta$ accordingly.

Caching and budgeting

Often, similar questions are asked, which can be used to answer one another, such as taking the sum, count, and later the mean. The net privacy loss can be minimised by caching and reusing the queries and responses across multiple queries.

Note

When setting a lifetime Privacy Budget for a dataset it is advised to choose $\delta$ not greater than the inverse of the dataset size e.g. for a dataset with million records, a delta value of $10^{-6}$.