LLM TracingAdvanced Features

Masking

Protect your sensitive information from traces using the masking feature

Overview

Masking allows you to automatically redact or transform sensitive data in your traces before they’re sent to the observatory. Masking is essential for several reasons:

  • Security: Prevent exposure of credentials or sensitive business data
  • Regulatory Compliance: Meet requirements like GDPR, HIPAA, or CCPA

Configure Masking

To implement masking, you need to define a masking function and configure the trace manager to use it.

main.py
1import re
2from deepeval.tracing import observe, trace_manager
3
4def masking_function(data):
5    if isinstance(data, str):
6        data = re.sub(r'\b(?:\d{4}[- ]?){3}\d{4}\b', '[REDACTED CARD]', data)
7        return data
8    return data
9
10trace_manager.configure(mask=masking_function)
11
12@observe()
13def llm_app(query: str):
14    return "4242-4242-4242-4242"
15
16llm_app("Test Masking")

The masking function is automatically applied to:

  1. Span attributes: Any attributes passed to span decorators or wrappers
  2. Observed function I/O: all input parameters and return values of functions with @observe decorator.

Since the masking function is applied to an observed function’s inputs, outputs, and span attributes, it must handle the various data types defined for each field as necessary.

Example Masking functions

Credit card number

main.py
1import re
2from typing import Any
3
4def mask_credit_card(data: Any) -> Any:
5    if isinstance(data, str):
6        return re.sub(r'\b(?:\d{4}[- ]?){3}\d{4}\b', '[REDACTED CARD]', data)
7    elif isinstance(data, list):
8        return [mask_credit_card(item) for item in data]
9    elif isinstance(data, dict):
10        return {k: mask_credit_card(v) for k, v in data.items()}
11    else:
12        return data
13
14
15print(mask_credit_card("My card number is 4111 1111 1111 1234."))
16# Output: My card number is [REDACTED CARD].

Email address

main.py
1import re
2from typing import Any
3
4def mask_email(data: Any) -> Any:
5    if isinstance(data, str):
6        return re.sub(r'\b([\w.%+-])[^\s@]*(@[\w.-]+\.\w+)', r'\1*****\2', data)
7    elif isinstance(data, list):
8        return [mask_email(item) for item in data]
9    elif isinstance(data, dict):
10        return {k: mask_email(v) for k, v in data.items()}
11    else:
12        return data
13
14print(mask_email("Contact me at johndoe@example.com."))
15# Output: Contact me at j*****@example.com.

Bank account number

main.py
1import re
2from typing import Any
3
4def mask_bank_account(data: Any) -> Any:
5    if isinstance(data, str):
6        return re.sub(
7            r'\b\d{6,}(?!\d)',
8            lambda m: '*' * (len(m.group()) - 4) + m.group()[-4:],
9            data
10        )
11    elif isinstance(data, list):
12        return [mask_bank_account(item) for item in data]
13    elif isinstance(data, dict):
14        return {k: mask_bank_account(v) for k, v in data.items()}
15    else:
16        return data
17
18
19print(mask_bank_account("My account is 9876543210."))
20# Output: My account is ******3210.

Passport number

main.py
1import re
2from typing import Any
3
4def mask_passport_number(data: Any) -> Any:
5    if isinstance(data, str):
6        return re.sub(r'\b([A-Z])([A-Z0-9]{6,8})([A-Z0-9])\b', r'\1*******\3', data)
7    elif isinstance(data, list):
8        return [mask_passport_number(item) for item in data]
9    elif isinstance(data, dict):
10        return {k: mask_passport_number(v) for k, v in data.items()}
11    return data
12
13print(mask_passport_number("Passport: A12345678."))
14# Output: Passport: A*******8.