Top 23 Python prompt-injection Projects

prompt-injection

• Add a project

1. iFixAi

   1 4 23,720 8.8 Python

   Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.

   

   Project mention: GitHub AI Repos ประจำสัปดาห์: สัปดาห์ที่ agent มีด่านตรวจของตัวเอง (7 ต.ค. 2026) | dev.to | 2026-10-10

   [1] GitHub Trending, weekly, 2026, https://github.com/trending?since=weekly [2] GitHub Trending, รายภาษา (Go, Python, TypeScript) weekly และ monthly, 2026, https://github.com/trending/go?since=weekly [3] GitHub Ranking AI, Top 100 AI Agents, 2026, https://raw.githubusercontent.com/yuxiaopeng/Github-Ranking-AI/main/Top100/AI%20Agents.md [4] NVIDIA/OpenShell, README, 2026, https://github.com/NVIDIA/OpenShell [5] NVIDIA OpenShell documentation, 2026, https://docs.nvidia.com/openshell/latest/index.html [6] NVIDIA/OpenShell, releases, v0.1.2, 28 กันยายน 2026, https://github.com/NVIDIA/OpenShell/releases [7] ifixai-ai/iFixAi, README, 2026, https://github.com/ifixai-ai/iFixAi [8] mvschwarz/openrig, README, 2026, https://github.com/mvschwarz/openrig [9] mvschwarz/openrig, releases, v0.6.5, 4 ตุลาคม 2026, https://github.com/mvschwarz/openrig/releases [10] cursor/plugins, README, 2026, https://github.com/cursor/plugins [11] anthropics/knowledge-work-plugins, README, 2026, https://github.com/anthropics/knowledge-work-plugins [12] JuliusBrussee/caveman, README, 2026, https://github.com/JuliusBrussee/caveman [13] Adeyemi, M. D., Rossi, R. A., Dernoncourt, F., "CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression", arXiv 2606.24083, 2026, https://arxiv.org/abs/2606.24083 [14] Shiryaev, D., "Does Speaking to Agents Like Cavemen Really Save 65% of Tokens? We Test", JetBrains AI Blog, 2026, https://blog.jetbrains.com/ai/2026/07/speak-to-ai-agents-like-cavemen-tosave-tokens/ [15] headroomlabs-ai/headroom, README, 2026, https://github.com/headroomlabs-ai/headroom [16] VectifyAI/PageIndex, README, 2026, https://github.com/VectifyAI/PageIndex [17] VectifyAI/PageIndex, releases, v0.2.21, 1 ตุลาคม 2026, https://github.com/VectifyAI/PageIndex/releases

2. AppSignal

   www.appsignal.com sponsored

   Monitoring that respects your time & budget. APM, error tracking, and dashboards for modern web apps. Ten-minute setup, transparent flat pricing, and support from engineers who actually use the product.

   

3. AI-Infra-Guard

   2 3 6,884 9.8 Python

   A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.

   

   Project mention: Black Hat Europe 2025 Arsenal: 8 AI Security Tools Transforming Cybersecurity | dev.to | 2025-11-09

   https://github.com/Tencent/AI-Infra-Guard https://github.com/mandiant/harbinger https://github.com/stratosphereips/MIPSEval https://github.com/ErdemOzgen/RedAiRange https://github.com/ReversecLabs/spikee https://github.com/ThalesGroup/sql-data-guard

4. ADR

   3 1 2,056 9.3 Python

   ADR secures enterprise AI agents through observability, security benchmarking, and threat detection. Deployed at Uber. (by uber)

   

   Project mention: Uber open-sourced its security monitoring for Claude Code, Cursor and Codex | news.ycombinator.com | 2026-08-06

5. promptmap

   4 2 1,281 6.6 Python

   a security scanner for custom LLM applications

   

6. agentdojo

   5 3 923 7.3 Python

   A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents.

   

   Project mention: AI agents can't tell who's giving the orders. So I built a tiny gate. | dev.to | 2026-09-23

   So I took 10 free, open-source detectors and ran them against 629 realistic attacks from AgentDojo, a research benchmark from ETH Zurich. Each attack was hidden inside normal-looking tool output, like a bill, an email or a hotel review, exactly the way an agent would see it.

7. HOL Guard

   6 0 850 10.0 Python

   Open-source antivirus for AI agents: block risky tools, secret access, prompt injection, malicious packages, MCP servers, plugins, and skills at runtime.

   

8. skylos

   7 7 846 9.8 Python

   Open source local-first PR scanner that finds dead code, security bugs, secrets, quality regressions, and AI-code mistakes before merge. For first timers refer to https://duriantaco.github.io/skylos/repo-map/

   

   Project mention: GitHub Actions Security and GitLab CI Security: Static Analysis for CI/CD | dev.to | 2026-05-12

   pip install "git+https://github.com/duriantaco/skylos.git" skylos . --danger

9. SaaSHub

   www.saashub.com sponsored

   SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

   

10. Open-Prompt-Injection

    8 1 506 2.8 Python

    This repository provides a benchmark for prompt injection attacks and defenses in LLMs

    

11. agentseal

    9 2 377 9.2 Python

    Security toolkit for AI agents. Scan your machine for dangerous skills and MCP configs, monitor for supply chain attacks, test prompt injection resistance, and audit live MCP servers for tool poisoning.

    

    Project mention: AgentSeal – Security scanner for AI agents (150 attack probes) | news.ycombinator.com | 2026-03-06

    GitHub: https://github.com/AgentSeal/agentseal

12. aegis

    10 4 269 5.6 Python

    Self-hardening firewall for large language models (by automorphic-ai)

    

13. humanbound

    11 7 192 9.3 Python

    Open-source adversarial testing engine, SDK, and CLI for AI agents. Runs locally or against the Humanbound Platform.

    

    Project mention: Google's Gemini breakout and what Google's own vulnerability data says about agents | dev.to | 2026-10-12

    Humanbound's engine, SDK and CLI are open source (Apache-2.0): star the repo and read the code at https://github.com/humanbound/humanbound. To run adversarial tests against your own agents, sign up for the free Community plan at https://app.humanbound.ai

14. diagnostic

    12 1 160 - Python

    iFixAi. The open-source diagnostic for AI misalignment. 32 tests across fabrication, manipulation, deception, unpredictability, and opacity. Provider-agnostic. Runs against OpenAI, Anthropic, Bedrock, Azure, Gemini, and more. Letter grade in under 5 minutes, content-addressed manifest for bit-identical replay. Built by iMe.

    

    Project mention: Open-source diagnostic for Al misalignment. Model agnostic, industry agnostic | news.ycombinator.com | 2026-05-04

15. AIGoat

    13 1 146 7.2 Python

    AIGoat - Open-source AI security playground for hands-on LLM red teaming. Explore prompt injection, RAG poisoning, AI agents, MCP security, and OWASP Top 10 for LLM Applications through practical attack labs, CTF challenges, and progressive defenses all running locally.

    

    Project mention: Introducing AIGoat 2.0: Hands-On Security Labs for LLMs, RAG, MCP, and AI Agents | dev.to | 2026-10-13

16. gateway

    14 1 131 7.6 Python

    The Open Source Firewall for LLMs. A self-hosted gateway to secure and control AI applications with powerful guardrails. (by trylonai)

    

17. localmod

    15 1 127 8.5 Python

    Self-hosted content moderation API that outperforms Amazon Comprehend. 100% offline, your data never leaves your server. Text + Image moderation.

    

    Project mention: I Built a Self-Hosted Content Moderation API (Open Source) | dev.to | 2026-01-01

    So I built LocalMod, a fully offline, self-hosted content moderation API. It handles both text and images, and actually outperforms some commercial services on toxicity benchmarks.

18. project_mantis

    16 2 126 1.8 Python

    Project Mantis: Hacking Back the AI-Hacker; Prompt Injection as a Defense Against LLM-driven Cyberattacks

    

19. kereva-scanner

    17 3 78 8.1 Python

    Code scanner to check for issues in prompts and LLM calls

    

20. jev-judge-mcp

    18 1 72 9.6 Python

    Typed judgment tools for MCP agents. TypeSafe's Jev model as verify, screen, find, classify, rerank, decide, compare, extract, review, gate, and score: the model judges, policy decides auto, review, or escalate.

    

    Project mention: An MCP server that gates/verifies/screens before agents act | news.ycombinator.com | 2026-09-30

21. open-bias

    19 2 71 - Python

    Open Source Agent Alignment: Make your agents follow rules. One line of code to‎ ‎enforce, trace, and improve. ‎ ‎

    

    Project mention: Show HN: Open Bias – proxy that enforces agent behavior at runtime | news.ycombinator.com | 2026-04-28

22. PDF-Prompt-Injection-Toolkit

    20 1 66 4.2 Python

    A red team / blue team toolkit for testing and detecting prompt injection attacks hidden inside PDF documents. 一个用于测试和检测 PDF 文档中隐藏的提示词注入攻击的红蓝对抗工具包。

    

    Project mention: PDF Prompt Injection Toolkit – Hidden LLM Payloads in PDFs | news.ycombinator.com | 2026-03-26

23. llmtrace

    21 1 53 9.7 Python

    Zero-code LLM security & observability proxy. Real-time prompt injection detection, PII scanning, and cost control for OpenAI-compatible APIs. Built in Rust.

    

    Project mention: We Fine-Tuned a 3B Model to Refuse Prompt Injections | dev.to | 2026-03-05

    We hit this wall ourselves. Together with George Politis, we've been running LLMTrace, an open-source security proxy that sits between applications and their LLM providers. It intercepts every request and runs it through an ensemble of detectors (regex patterns, a DeBERTa classifier, InjecGuard, jailbreak classifiers) at ~50ms overhead on the hot path. On known jailbreak datasets it hits 99% recall. We were reasonably confident in it until we ran 12,000+ adversarial prompts against it and watched 498 attacks sail through. Most of the damage came from the SaTML CTF corpus, competition-grade prompts designed specifically to beat detectors, which dropped our recall to 92%. Social engineering wrapped in polite language, indirect injections buried in data payloads. The pattern matchers hadn't seen any of it.

24. buried-injections

    22 2 30 4.2 Python

    🛡️ Regex catches 0%, Meta's Prompt Guard 2 catches 1% of 629 realistic AgentDojo injection attacks when they're buried in tool output. Reproducible benchmark.

    

    Project mention: Your AI Agent Will Follow a Stranger's Instructions. Here's How I Actually Test For It. | dev.to | 2026-09-26

    git clone https://github.com/rudratoshs/buried-injections cd buried-injections make setup make bench-agentdojo

25. rag-redteam

    23 1 29 8.5 Python

    Red-team your RAG pipeline for prompt injection, source-document leakage, and cross-document smuggling, in CI. The gap between RAG eval (RAGAS/DeepEval) and LLM scanners (garak).

    

    Project mention: RAG-redteam, red-team your RAG pipeline for injection and leakage in CI | news.ycombinator.com | 2026-07-08

26. SaaSHub

    www.saashub.com sponsored

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    

Python prompt-injection related posts

• AI agents can't tell who's giving the orders. So I built a tiny gate.

  1 project | dev.to | 23 Sep 2026

• Designing an eval harness for prompt-injection detection: what measuring my defenses actually taught me

  4 projects | dev.to | 21 Sep 2026

• Sentinel-scan-CLI – a CLI that runs 15 prompt-injection attacks against your LLM

  1 project | news.ycombinator.com | 25 Aug 2026

• I tested whether agents leak secrets without being asked — and whether my scanner catches it when they do

  1 project | dev.to | 10 Aug 2026

• agent-shield-runtime v0.1.0: hook de despliegue que conecta 5 sensores de defensa de agentes IA

  6 projects | dev.to | 20 Jul 2026

• 0.2.0: I shipped the coverage my own page had already promised

  1 project | dev.to | 18 Jul 2026

• Show HN: Interlock – Detect post-approval MCP tool drift before execution

  1 project | news.ycombinator.com | 11 Jul 2026
• A note from our sponsor - SaaSHub
  www.saashub.com | 16 Oct 2026

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

Monitoring that respects your time & budget

APM, error tracking, and dashboards for modern web apps. Ten-minute setup, transparent flat pricing, and support from engineers who actually use the product.

www.appsignal.com

Do not miss the trending Python projects with our weekly report!