My Security Stack for Open Source AI Projects

Photo by Kevin Ku on Unsplash

---

I made Orbit AI public this month. The code did not change overnight, but the risk did.

Orbit AI is CRM infrastructure for AI agents and developers: a REST API, TypeScript SDK, CLI, MCP server, and connectors for Gmail, Calendar, and Stripe. It is not a hosted SaaS product. It is infrastructure someone can deploy next to their own app.

That changes the job of the repository. It is no longer just source code. It is the product surface people inspect before they trust you: the dependencies, CI, permissions, security policy, contribution path, and maintainer decisions.

When a project is private, too much security context can stay in your head. You know who has access. You know which script is experimental. You know which failing test is harmless and which one means stop. When a project becomes public, that private context has to move into the repo. A contributor should be able to understand how changes are reviewed. A user should be able to report a vulnerability without guessing. An AI agent should be able to open a pull request without accidentally bypassing the process.

I have been building a lot in this category: CheckApp, Jobot, Orbit AI, and the Open Agents deploy guide. They are different products, but they share the same risk pattern. They move fast, pull in many packages, depend on automation, and increasingly get changed by agents that can install dependencies, edit files, run tests, and open PRs.

That is why I stopped treating security as a checklist I would clean up later. For open-source AI projects, security is part of the launch.

---

The Security Problem I Was Actually Solving

The obvious problem is dependency risk. AI projects pull in model SDKs, auth libraries, vector database clients, scraping tools, browser automation, payment SDKs, telemetry, workflow engines, and deployment helpers. A modern TypeScript AI app becomes a supply-chain project almost by accident.

But dependencies are only one layer. The broader issue is that open-source AI projects have more places where code can run than most teams account for:

• package install hooks
• GitHub Actions workflows
• code generated by AI agents
• MCP tools and local helper scripts
• dependency update PRs
• one-off setup commands copied from documentation
• release and publishing workflows

Reviewing only the application code is not enough. The repo is a small operating environment with permissions, automation, credentials, package managers, external apps, and humans making fast decisions under pressure.

The last few weeks made that concrete. In April 2026, Vercel disclosed a security incident that started with a compromised third-party AI tool used by an employee. Vercel said it worked with GitHub, Microsoft, npm, and Socket to verify that Vercel npm packages had not been compromised. The lesson was not "avoid AI tools." That is not realistic. The lesson is that AI tools now sit inside the trust boundary of software development.

The npm ecosystem had its own reminder. In March 2026, malicious Axios releases were published to npm after a maintainer account was compromised, according to reporting on the Axios npm compromise. The important detail is install-time behavior. In that kind of attack, the damage can happen when the package is installed, not only when application code imports it.

CI/CD is part of the same story. In March 2025, CISA warned about the compromise of tj-actions/changed-files, a widely used GitHub Action that could expose secrets from workflow logs. That changes how you look at third-party actions, workflow permissions, and which secrets are available to automation.

So the question for Orbit AI became practical: what is the smallest serious security stack I can use for open-source AI projects without turning every PR into an enterprise process?

---

The Baseline: GitHub Has to Be Locked First

Before adding more tools, I started with the GitHub baseline. This is the part that feels obvious until you skip it.

For a public AI repo, I want:

• protected main
• pull requests required before merge
• required CI checks
• only trusted maintainers able to merge
• secret scanning and push protection
• Dependabot alerts
• CodeQL or equivalent code scanning
• least-privilege GitHub Actions permissions
• a SECURITY.md disclosure path
• clear contribution rules

GitHub documents these capabilities in its security features guide. For public repositories, a lot of the useful protection is available without buying another product.

This layer decides whether everything else matters. If main is not protected, a dependency scanner is only advisory. If CI is optional, a pull request can merge with a broken build. If secrets can be pushed, every later tool is cleanup. If GitHub Actions have broad permissions by default, a compromised workflow has more room to move.

For Orbit AI, this became the first rule: the public repo needs a clear merge path. Humans and agents can contribute, but only maintainers should merge, and the merge should go through required checks.

---

Why I Added Socket

The next layer is Socket.

I did not add Socket because I wanted another vulnerability dashboard. I added it because traditional vulnerability scanning does not fully answer the supply-chain question I care about.

A package can have no known CVE and still be suspicious. It can add an install script, touch the network, access environment variables, run shell commands, introduce native code, or look like a typosquat.

That behavior matters in AI projects because dependency changes happen often. Agents may suggest a new package. A contributor may add an SDK. Dependabot may open a version bump. A maintainer may approve a small-looking dependency update because the application diff looks harmless.

Socket is useful because it analyzes dependency behavior and puts feedback close to the decision point. Their GitHub integration is built around pull requests, and their Socket for GitHub documentation explains how dependency manifest changes trigger alerts and comments.

The way I want to use Socket at first is deliberately conservative:

• start non-blocking
• review alerts on dependency changes
• pay special attention to install scripts and shell execution
• promote only high-confidence findings to hard blockers later

That last point matters. Security tools lose credibility when they block too much too early. For a public open-source repo, I want Socket to give me visibility first. Once I understand the signal quality in this codebase, I can decide what should block a merge.

---

Why I Added DepsGuard

The third layer is DepsGuard, the open-source package-manager hardening tool from Arnica.

DepsGuard solves a different problem from Socket. Socket helps me see suspicious dependency behavior. DepsGuard helps reduce the chance that a freshly poisoned package version gets installed in the first place.

The idea is simple: many supply-chain attacks have a short active window. A malicious version gets published, installers pull it quickly, the ecosystem notices, and the version gets removed. If your package manager immediately accepts every brand-new version, your local machine or CI can land inside that window. If your environment waits before accepting very new releases, you buy time for the ecosystem to catch the problem.

DepsGuard checks whether package manager controls like release-age delays and install-script restrictions are enabled. Its own documentation at depsguard.com explains the supported settings for npm, pnpm, Yarn, and bun, including minimum release age and install script controls.

This is why I like DepsGuard: it is narrow. It does not pretend to replace SCA, secret scanning, CodeQL, or human review. It focuses on one layer of the problem: package installation behavior.

The Orbit AI scan also produced a useful planning lesson. Some stronger pnpm hardening controls require newer pnpm versions than the repo currently uses. The responsible move is not to hide a package-manager migration inside a security PR. The responsible move is to open a dev-toolchain migration issue, explain why it matters, and handle it separately.

That distinction is important in open source. Security work should be fast when the fix is small and safe. It should be planned when the fix changes the toolchain.

---

Why I Am Not Starting With Full Arnica Everywhere

Arnica is worth separating from DepsGuard. DepsGuard is the small open-source tool. Arnica is a broader AppSec platform.

The full Arnica product is more interesting when you have an organization with multiple repositories, multiple teams, and a real need for security posture management. It can help with repository visibility, ownership mapping, SAST, SCA, secrets, IaC, and developer-native remediation workflows. Arnica's GitHub integration page describes the broader platform, and its GitHub App permissions documentation is the page I would review before installing it broadly.

For one or two public repos, I would not start there. The permission surface is bigger, the rollout requires more governance, and the value depends on organization-wide visibility. That does not mean Arnica is bad. It means the tool should match the size of the problem.

My current view:

• for a single public repo, start with GitHub native security, Socket, DepsGuard, CodeQL, and Dependabot
• for an organization with many repos, evaluate Arnica alongside GitHub Advanced Security, Endor Labs, Semgrep, Snyk, and Socket's enterprise offering
• choose based on signal quality, developer workflow, permission model, ownership mapping, and how well it turns findings into fixes

That is the distinction I care about. Arnica becomes compelling when the question is not "is this repo safe?" but "do we understand risk across all our repos, owners, branches, dependencies, and developer workflows?"

---

The Stack I Am Using Now

Here is the stack I am comfortable standardizing across my open-source AI projects right now.

The important thing is that these tools do not all do the same job. Dependabot is not Socket. Socket is not DepsGuard. CodeQL is not secret scanning. Branch protection is not dependency security. The stack works because each layer covers a different failure mode.

It also keeps the public explanation clean. If someone asks why a tool is installed, I can explain the specific job it has. That matters because open-source trust is not only technical. It is procedural.

---

The CI Tradeoff

The other thing I had to revisit was CI.

I do not want a tiny CI setup that misses real breakage. I also do not want every PR to run a huge test matrix that makes contributors and agents wait forever.

For AI-assisted development, slow CI becomes a security problem in a different way: people start avoiding it, splitting changes awkwardly, or treating red checks as background noise.

The better model is risk-based CI:

• normal PRs run lint, typecheck, focused tests, and content validation
• dependency or workflow changes get dependency/security review
• changes touching auth, secrets, CI, publishing, MCP tools, or tenant isolation get deeper human review
• scheduled jobs run heavier suites that do not need to block every small PR
• release branches run the full set of checks before publishing

This is the balance I want for Orbit AI. Security should make the project safer without making normal maintenance impossible. A README typo and a change to pnpm-workspace.yaml should not be treated as the same risk.

---

The Copy/Paste Version

If you are making an AI project public, I would not start with a giant vendor evaluation. I would start with the repo itself.

Before announcing the project, I would make sure:

1. main is protected.
2. Pull requests are required before merge.
3. CI is required and fast enough to respect.
4. Secret scanning and push protection are enabled.
5. Dependabot alerts are enabled.
6. CodeQL or equivalent code scanning is running.
7. GitHub Actions permissions are least-privilege.
8. Socket is installed in advisory mode for dependency behavior.
9. DepsGuard has been run read-only and the findings are converted into issues.
10. SECURITY.md explains how to report vulnerabilities.

After that, document the tradeoffs. If Socket is non-blocking, say that. If DepsGuard recommends a control that requires a pnpm upgrade, open an issue instead of hiding it in a rushed PR. If a tool is advisory, be clear about what still requires human review.

That is the part I think open-source projects often miss. Security is not only the set of tools you install. It is the explanation of how maintainers make decisions when tools disagree, fail, or produce noise.

---

What Changed for Orbit AI

The practical outcome for Orbit AI is not one big security rewrite. It is a maintainable process.

The repo now has a clearer public security philosophy: protect the default branch, keep dependency behavior visible, slow down risky installs, keep CI useful, and document the decisions that cannot be safely automated yet.

The next work is operational: keep reviewing Socket PR comments, turn DepsGuard findings into package-manager hardening issues, keep dev-toolchain migrations separate from urgent security fixes, and make sure only trusted maintainers can merge.

That is not as flashy as saying "install this one product and you are covered." But it is more honest. Open-source AI security is not a single product decision. It is a set of small, layered controls that make the repo harder to compromise and easier to maintain in public.

The real lesson is that AI changes the speed of software work before it changes the fundamentals of security. Agents can write code faster, but the repo still needs protected branches. Dependencies can be added faster, but package installation still needs guardrails. CI can be triggered more often, but the checks still need to be meaningful.

The faster the workflow gets, the more important it becomes to make the trust boundaries explicit.

That is the security model I want for my open-source AI projects: simple enough to run every day, visible enough for contributors to understand, and layered enough that one missed signal does not become the whole defense.