Waiting for input...
Star SPIKE on GitHub

ADR-0019: Plugin-Based Storage Backend Architecture


Context

Following ADR-0014’s decision to maintain SQLite as SPIKE’s primary storage backend, we recognize the need for flexibility in storage solutions for different deployment scenarios. Some deployments may require:

  • Alternative storage backends for enterprise environments
  • Specialized storage solutions for specific use cases
  • Custom implementations for unique requirements
  • In-memory storage for development and testing

Decision

We will implement a plugin-based architecture for storage backends while maintaining SQLite as the default implementation. This architecture will:

  1. Define a clear storage interface that all backends must implement
  2. Provide official plugins for common use cases
  3. Enable third-party plugin development
  4. Maintain backward compatibility with existing SQLite implementations

Rationale

Plugin System Benefits

1. Extensibility:

  • Users can implement custom storage solutions
  • Third-party developers can contribute new backends
  • Specialized solutions can be developed without modifying core code

2. Maintainability:

  • Core codebase remains focused on primary functionality
  • Plugins can be maintained independently
  • Testing and validation can be scoped to specific implementations

3. Deployment Flexibility:

  • Different environments can use appropriate storage solutions
  • Migration between backends becomes possible
  • Development and testing can use simplified implementations

Official Plugin Scope

1. Core Implementation:

  • SQLite (default backend, ships with core)
  • In-memory (for development and testing)

2. Additional Official Plugins:

  • PostgreSQL
  • S3/MinIO compatible storage

Interface Requirements

Storage plugins must implement:

1. Core Operations:

  • CRUD operations for secrets
  • Atomic transactions
  • Concurrent access handling
  • Error handling and recovery

2. Security Features:

  • Access control integration
  • Audit logging support

3. Management Functions:

  • Health checks
  • Backup/restore capabilities
  • Migration tools

Implementation

Plugin Architecture

Here is a sample structure for the plugin architecture. Note that this is a starting point and it will likely be subject to change.

// Interface
type StorageBackend interface {
    // Core operations
    Create(ctx context.Context, secret *Secret) error
    Read(ctx context.Context, id string) (*Secret, error)
    Update(ctx context.Context, secret *Secret) error
    Delete(ctx context.Context, id string) error
    
    // Management
    Initialize(ctx context.Context, config *Config) error
    Healthcheck(ctx context.Context) error
    
    // Transaction support
    BeginTx(ctx context.Context) (Transaction, error)
}

// Registration
type PluginRegistry interface {
    Register(name string, factory StorageFactory) error
    Get(name string) (StorageFactory, error)
}

Consequences

Positive

  1. Increased flexibility for different deployment scenarios
  2. Clear path for community contributions
  3. Simplified testing with in-memory implementation
  4. Better separation of concerns

Negative

  1. Additional complexity in plugin management
  2. Need for plugin validation and security review
  3. Potential for incompatible plugin implementations
  4. Documentation overhead for multiple backends

Mitigations

  1. Plugin Validation:
  • Provide comprehensive test suites
  • Implement plugin verification tools
  • Document security requirements
  1. Compatibility:
  • Version plugin interface explicitly
  • Provide migration tools between backends
  • Maintain compatibility tests
  1. Documentation:
  • Clear plugin development guidelines
  • Example implementations
  • Best practices documentation