PromptShield security report

Treat user text as data, and enforce policy outside the model.

• Keep instructions and user input in separate roles; never concatenate them into one string.
• Run an input classifier (or at minimum normalisation + decoding of base64/unicode) before the model sees the message.
• Validate the *output* against what the app is allowed to say, not just the input.
• Don't rely on 'ignore attempts to override these rules' in the prompt; it is not a control.

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": sanitize(user_text)},  # never f"{SYSTEM_PROMPT}{user_text}"
]
reply = llm(messages)
if not policy_check(reply):  # deterministic allow-list / classifier
    reply = "Sorry, I can't help with that."

Treat user text as data, and enforce policy outside the model.

• Keep instructions and user input in separate roles; never concatenate them into one string.
• Run an input classifier (or at minimum normalisation + decoding of base64/unicode) before the model sees the message.
• Validate the *output* against what the app is allowed to say, not just the input.
• Don't rely on 'ignore attempts to override these rules' in the prompt; it is not a control.

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": sanitize(user_text)},  # never f"{SYSTEM_PROMPT}{user_text}"
]
reply = llm(messages)
if not policy_check(reply):  # deterministic allow-list / classifier
    reply = "Sorry, I can't help with that."

Treat user text as data, and enforce policy outside the model.

• Keep instructions and user input in separate roles; never concatenate them into one string.
• Run an input classifier (or at minimum normalisation + decoding of base64/unicode) before the model sees the message.
• Validate the *output* against what the app is allowed to say, not just the input.
• Don't rely on 'ignore attempts to override these rules' in the prompt; it is not a control.

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": sanitize(user_text)},  # never f"{SYSTEM_PROMPT}{user_text}"
]
reply = llm(messages)
if not policy_check(reply):  # deterministic allow-list / classifier
    reply = "Sorry, I can't help with that."

Model output is untrusted input to your next component. Encode it for its sink.

• HTML-escape model output before rendering; render markdown with a sanitizer (no raw HTML, no javascript: URLs).
• Strip or proxy external images/links so the model cannot exfiltrate data via URLs.
• Never pass model output to SQL/shell/eval; use parameterised queries and allow-listed commands.

import html, re
def safe_render(reply: str) -> str:
    reply = re.sub(r'!\[[^\]]*\]\(https?://[^)]*\)', '[image removed]', reply)
    return html.escape(reply)

Don't let a persona change the rules. Policy must not live in the prompt alone.

• Detect and refuse persona-switch / 'no rules' framing before it reaches the model.
• Pin the role in the system message and re-state constraints after long user turns.
• Run a second-pass moderation check on the output; a jailbroken model still produces output you can inspect.

PERSONA_RE = re.compile(r'(you are now|developer mode|no (rules|restrictions)|pretend)', re.I)
if PERSONA_RE.search(user_text):
    return "I can't switch roles, but I'm happy to help with your order."

Treat user text as data, and enforce policy outside the model.

• Keep instructions and user input in separate roles; never concatenate them into one string.
• Run an input classifier (or at minimum normalisation + decoding of base64/unicode) before the model sees the message.
• Validate the *output* against what the app is allowed to say, not just the input.
• Don't rely on 'ignore attempts to override these rules' in the prompt; it is not a control.

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": sanitize(user_text)},  # never f"{SYSTEM_PROMPT}{user_text}"
]
reply = llm(messages)
if not policy_check(reply):  # deterministic allow-list / classifier
    reply = "Sorry, I can't help with that."