When building a secrets management system, one of the key challenges is providing secure automated access for CI/CD pipelines while maintaining strong security boundaries. In this post, we’ll explore different architectural approaches to solving this problem, analyzing their trade-offs and providing recommendations.

The Base Architecture

Before diving into the solutions, let’s understand our starting point. The existing system consists of two main components:

  1. Pilot: A CLI tool that provides user access to secrets
  2. Nexus: The secure storage backend for secrets

The communication between these components uses SPIFFE mTLS over REST, providing strong security guarantees and identity-based access control.

The Challenge

While this architecture works well for human operators using the CLI, we need to provide automated access for CI/CD pipelines. This requires exposing a RESTful interface that can be consumed by automation tools while maintaining our security posture.

Solution Options

1. Extend Pilot as an OIDC Resource Server

In this approach, we would enhance the existing Pilot component to serve as an OIDC resource server, allowing it to handle automated requests alongside its CLI duties.

Advantages:

  • Reuses existing, trusted component
  • Maintains current security boundary where only Pilot talks to Nexus
  • Preserves existing SPIFFE mTLS security model
  • Reduces operational complexity by not adding new components

Disadvantages:

  • Mixes different concerns in one component
  • Complicates Pilot’s codebase and testing
  • May require significant architectural changes to Pilot
  • Could impact CLI performance and reliability
  • Makes future changes more complex

2. Create a Dedicated OIDC Resource Server

This solution involves building a new component specifically designed to handle CI/CD automation requests, essentially acting as a Pilot analog for automated access.

Advantages:

  • Clean separation of concerns
  • Purpose-built for CI/CD integration needs
  • Independent scaling and maintenance
  • Can implement specific rate limiting and security controls
  • Separate logging and auditing for automated access
  • Easier to evolve and maintain
  • Can be optimized for automated workflows

Disadvantages:

  • New component to maintain and monitor
  • Need to implement SPIFFE mTLS client functionality
  • Additional operational overhead
  • Initial development investment

3. Expose Nexus as an OIDC Resource Server

The third option involves modifying Nexus to directly handle automated requests as an OIDC resource server.

Advantages:

  • Potentially simpler architecture
  • Direct access to secrets store
  • Fewer network hops

Disadvantages:

  • Breaks single-responsibility principle
  • Increases attack surface of critical secrets store
  • Complicates Nexus’s security model
  • Makes storage backend changes more difficult
  • Violates the principle of least privilege

After careful analysis, creating a dedicated OIDC resource server (Option 2) emerges as the most robust solution. Here’s why:

Security Benefits

  • Maintains existing security boundaries
  • Allows for fine-grained access control specific to automation
  • Keeps Nexus protected behind SPIFFE mTLS
  • Enables separate security policies for automated access

Architectural Benefits

  • Clean separation of concerns
  • Independent scaling and evolution
  • Dedicated logging and monitoring
  • Easier to maintain and modify

Operational Benefits

  • Can implement automation-specific features
  • Independent deployment and updates
  • Separate rate limiting and throttling
  • Dedicated audit trails for automated access

Implementation Considerations

When implementing the dedicated OIDC resource server approach, consider:

  1. Authentication:

    • Implement robust OIDC client authentication
    • Consider additional authentication methods for different CI/CD platforms
    • Implement proper token validation and management

  2. Authorization:

    • Define clear RBAC policies for CI/CD access
    • Implement fine-grained access control
    • Consider environment-based restrictions

  3. Security:

    • Implement rate limiting
    • Add robust audit logging
    • Consider request validation and sanitization
    • Implement proper error handling that doesn’t leak sensitive information

  4. Monitoring:

    • Add detailed operational metrics
    • Implement automated access pattern analysis
    • Set up alerting for suspicious activities

Conclusion

While all three approaches could work, creating a dedicated OIDC resource server provides the best balance of security, maintainability, and flexibility. It allows for proper separation of concerns while maintaining strong security boundaries and enables independent evolution of human and automated access patterns.

This solution might require more initial development effort, but the long-term benefits in terms of security, maintainability, and operational flexibility make it the most robust choice for production environments.