SmartNoise Synth

Overview

SmartNoise-Synth is a privacy-preserving synthetic data generation tool developed by OpenDP. It uses advanced algorithms to generate realistic synthetic data while preserving privacy and ensuring statistical accuracy, making it valuable for research and analysis in sensitive domains like healthcare and finance.

Antigranular wraps the snsynth library to support PrivateDataframe with minimal change to the existing function signatures and methods. As a result, the transition is as smooth as possible for those who have worked with SmartNoise-Synth before.

To use the SmartNoise-Synth, you need to import it as presented in the following code block:

%%ag
import op_snsynth

1

Loading data

In this step, we download the pumps dataset and upload it to the session. Then, we create a Synthesizer and fit, to later create a sample.

import pandas as pd

# Load a dataset locally
pums = pd.read_csv("https://content.antigranular.com/image/notebook_content/PUMS.csv")

dataframe cached to server, loading to kernel...
DataFrame loaded successfully to the kernel

# Preview the non-private data
pums

    age	sex	educ	race	income	married
0	59	1	9	1	0.0	1
1	31	0	1	3	17000.0	0
2	36	1	11	1	0.0	1
3	54	1	11	1	9100.0	1
4	39	0	5	3	37000.0	0
...	...	...	...	...	...	...
995	73	0	3	3	24200.0	0
996	38	1	2	3	0.0	0
997	50	0	13	1	22000.0	1
998	44	1	14	4	500.0	1
999	29	1	11	1	66400.0	0

# Import it to AG
session.private_import(data=pums, name="pums")

2

Using a synthesizer

%%ag
from op_snsynth import Synthesizer

# Build a Synthesizer
synth = Synthesizer.create("mwem", epsilon=1.0, iterations=6, verbose=True)

# Fit it
fit = synth.fit(pums, preprocessor_eps=0.1)

    Processing 1 histograms

    Histogram #0 split: [0 1 2 3 4 5]
    Columns: 6
    Dimensionality: 308,352
    Cuboids possible: 63
    1-2-way cuboids possible: 21
    Fitting for 6 iterations
    Number of queries: 12
    Number of slices in queries: 2639
    Per-Measure Epsilon: 0.083
    Measurement Error: 27.63


    [0] - Average error: 1.384. Selected 6 slices

    [1] - Average error: 0.672. Selected 16 slices

    [2] - Average error: 0.496. Selected 96 slices

    [3] - Average error: 0.439. Selected 12 slices

    [4] - Average error: 0.366. Selected 803 slices

    [5] - Average error: 0.379. Selected 2 slices

The SmartNoise Synthesizers provide a variety of synthesizers for users to explore, other than MWEM. Here's a brief overview of each:

Marginal Synthesizers:

1

AIM

A synthesizer focusing on producing synthetic data while maintaining the marginal distributions of the original dataset.

2

MST

Another marginal synthesizer that focuses on maintaining statistical properties of the data.

3

PAC-Synth

A synthesizer designed to offer probably approximately correct (PAC) guarantees on the synthetic data generated.

Neural Network Synthesizers:

1

DP-CTGAN

A differentially private version of the Conditional Tabular Generative Adversarial Network (CTGAN).

2

PATE-CTGAN

An extension of CTGAN, which incorporates Private Aggregation of Teacher Ensembles (PATE) for enhanced privacy.

3

PATE-GAN

Utilizes the PATE framework in a GAN setting to generate synthetic data with privacy guarantees.

4

DP-GAN

A differentially private Generative Adversarial Network tailored for generating synthetic datasets.

Hybrid Synthesizers:

1

QUAIL

A hybrid synthesizer that combines various techniques to balance between data utility and privacy.

Each of these synthesizers offers unique approaches to generating synthetic data while addressing different aspects of privacy and data utility.

3

Exporting a sample

%%ag
# Get a sample of the same size as the dataset
sample = synth.sample(len(pums))

%%ag
# Export the sample to the local enviroment
export(sample, "sample")

Setting up exported variable in local environment: sample

# Preview of the data
sample

age	sex	educ	race	income	married
0	50	0	13	1	38502.4	1
1	60	1	12	1	23756.8	1
2	76	0	14	1	48332.8	1
3	40	0	9	1	43417.6	0
4	48	0	9	1	58163.2	0
...	...	...	...	...	...	...
995	37	0	13	1	67993.6	0
996	79	1	13	1	18841.6	1
997	52	1	11	2	53248.0	0
998	67	0	7	1	53248.0	0
999	36	1	11	1	28672.0	1

4

Comparing to real data (Ridge Classifier)

In this step, we created a function to compare the real versus synthetic data we have just created. We test it both with Ridge and Logistic Regression Classifiers.

from sklearn.metrics import classification_report, accuracy_score
from sklearn.model_selection import train_test_split

def test_real_vs_synthetic_data(real, synthetic, model):

# Create a DataFrame for synthetic data with the same columns as the real data
synth_df = pd.DataFrame(synthetic, columns=real.columns)

# Split real and synthetic datasets into features and labels
X = real.iloc[:, :-1]
y = real.iloc[:, -1]
X_synth = synth_df.iloc[:, :-1]
y_synth = synth_df.iloc[:, -1]

# Split both real and synthetic data into training and test sets
x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
x_train_synth, x_test_synth, y_train_synth, y_test_synth = train_test_split(X_synth, y_synth, test_size=0.2, random_state=42)

# Train model on real data
model_real = model()
model_real.fit(x_train, y_train)

# Train model on synthetic data
model_fake = model()
model_fake.fit(x_train_synth, y_train_synth)

# Evaluate and print performance of model trained on real data
predictions = model_real.predict(x_test)
print("\nTrained on Real Data:")
print(classification_report(y_test, predictions))
print("Accuracy real:", accuracy_score(y_test, predictions))

# Evaluate and print performance of model trained on synthetic data
predictions = model_fake.predict(x_test_synth)
print("\nTrained on Synthetic Data:")
print(classification_report(y_test_synth, predictions))
print("Accuracy synthetic:", accuracy_score(y_test_synth, predictions))

# Evaluate and print performance of random guessing
print("\nRandom Guessing:")
guesses = np.random.randint(0, (max(y_test_synth) - min(y_test_synth) + 1), len(y_test_synth))
np.random.shuffle(guesses)
print(classification_report(y_test_synth, guesses))
print("Accuracy guessing:", accuracy_score(y_test_synth, guesses))

return model_real, model_fake

# Let us start by testing a Ridge Classifier
from sklearn.linear_model import RidgeClassifier
import numpy as np

# We will compare the real dataset `pums` and the synthetic dataset `sample`
# using a Ridge Classifier
test_real_vs_synthetic_data(pums, sample, RidgeClassifier)

    Trained on Real Data:
                precision    recall  f1-score   support

            0       0.60      0.51      0.55        91
            1       0.63      0.72      0.67       109

        accuracy                           0.62       200
    macro avg       0.62      0.61      0.61       200
    weighted avg       0.62      0.62      0.62       200

    Accuracy real: 0.62

    Trained on Synthetic Data:
                precision    recall  f1-score   support

            0       0.52      0.11      0.18       104
            1       0.48      0.90      0.63        96

        accuracy                           0.48       200
    macro avg       0.50      0.50      0.40       200
    weighted avg       0.50      0.48      0.39       200

    Accuracy synthetic: 0.485

    Random Guessing:
                precision    recall  f1-score   support

            0       0.48      0.47      0.47       104
            1       0.43      0.44      0.44        96

        accuracy                           0.46       200
    macro avg       0.45      0.45      0.45       200
    weighted avg       0.46      0.46      0.46       200

    Accuracy guessing: 0.455

5

Comparing to real data (Logistic Regression)

# We can also test it with different classifier
from sklearn.linear_model import LogisticRegression
test_real_vs_synthetic_data(pums, sample, LogisticRegression)

    Trained on Real Data:
                precision    recall  f1-score   support

            0       0.63      0.35      0.45        91
            1       0.60      0.83      0.70       109

        accuracy                           0.61       200
    macro avg       0.62      0.59      0.57       200
    weighted avg       0.61      0.61      0.59       200

    Accuracy real: 0.61

    Trained on Synthetic Data:
                precision    recall  f1-score   support

            0       0.00      0.00      0.00       104
            1       0.48      1.00      0.65        96

        accuracy                           0.48       200
    macro avg       0.24      0.50      0.32       200
    weighted avg       0.23      0.48      0.31       200

    Accuracy synthetic: 0.48

    Random Guessing:
                precision    recall  f1-score   support

            0       0.49      0.44      0.47       104
            1       0.46      0.51      0.48        96

        accuracy                           0.48       200
    macro avg       0.48      0.48      0.47       200
    weighted avg       0.48      0.47      0.47       200

    Accuracy guessing: 0.475

6

Comparing data in a plot

We use a T-distributed stochastic neighbor embedding plot to display data from the real dataset and our synthetic dataset.

import matplotlib.patches as mpatches
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt

def tsne_plot(real, synthetic):

    # Prepare DataFrame for synthetic data
    synth_df = pd.DataFrame(synthetic, columns=real.columns)

    # Select features from real and synthetic data for t-SNE visualization
    x_train = real.iloc[:, :-1]
    x_train_synth = synth_df.iloc[:, :-1]

    # Combine and apply t-SNE to the data
    comb = np.vstack((x_train[:500], x_train_synth[:500]))
    embedding_1 = TSNE(n_components=2, perplexity=5.0, early_exaggeration=1.0).fit_transform(comb)
    x, y = embedding_1.T

    # Splitting the transformed data into two parts for plotting
    l = int(len(x) / 2)

    # Set up the plot
    plt.rcParams["figure.figsize"] = (8,8)
    plt.scatter(x, y, c=['red' for _ in range(l)] + ['blue' for _ in range(l)])
    red_patch = mpatches.Patch(color='red', label='Real Data')
    blue_patch = mpatches.Patch(color='blue', label='Synthetic Data')

    plt.legend(handles=[red_patch, blue_patch])

    plt.title('TSNE Plot, Real Data vs. Synthetic')
    plt.show()

# Plotting non-private and synthetic data
tsne_plot(pums, sample)

With this graph you can review how the synthetic data is distributed in comparison to the real data.

7

Extra: Using the MST Method

The acronym MST stands for “Maximum-Spanning-Tree” as the method produces differentially private synthetic data by relying on a “Maximum-Spanning-Tree" of mutual information.

MST finds the maximum spanning tree on a graph where nodes are data attributes and edge weights correspond to approximate mutual information between any two attributes.

We say approximate because the “maximum spanning tree” is built using the exponential mechanism, which helps select edge weights with high levels of mutual information in a differentially private manner. On the other hand, the marginals are measured using the Gaussian mechanism.

%%ag

df = pums.drop(["income"], axis=1)
df = df.sample(frac=1, random_state=42)

%%ag
mst_synth = Synthesizer.create("mst", epsilon=3.0, verbose=True)
sample = mst_synth.fit(df, preprocessor_eps=0.1)

%%ag
sample = mst_synth.sample(len(df))

8

Extra: What If We Use Data Transformers?

Even with preprocessing hints, the preprocessor inferred by the synthesizer may not be exactly what you want. For example, the mwem synthesizer will automatically bin continuous columns into 10 bins.

Spending epsilon to infer bounds is wasteful and reduces accuracy when you already have public bounds for continuous columns. In most cases, you will get the best performance by manually specifying the preprocessor you want to use.

Preprocessing is done by a TableTransformer object, which implements a differentially private reversible data transform.

%%ag
from op_snsynth import TableTransformer

# The create method can infer the min and max values from the data set. Inferring the min and max requires some privacy budget, specified by the epsilon argument.
tt = TableTransformer.create(pums, style='cube', categorical_columns=list(pums.columns), continuous_columns=['age', 'income'])

%%ag
pums_encoded = tt.fit_transform(pums, epsilon=3.0)
pums_decoded = tt.inverse_transform(pums_encoded)

%%ag
synth = Synthesizer.create("mwem", epsilon=1.0, iterations=6, verbose=True)
fit = synth.fit(pums_decoded, preprocessor_eps=0.1)

%%ag
sample = synth.sample(len(df))

Resources

You can access the official Smartnoise-Synth page to learn more about all the function signatures and methods.